Professional Documents
Culture Documents
Creating a reliable and repeatable process for releasing software can be achieved in phases, which helps reduce complexity and upfront investment of resources. Some areas will be simpler to automate than others. In addition, organizations will benefit by taking small steps toward Continuous Delivery even before the entire process is completely automated and the end-state is reached where production deployment is as simple as pressing a deploy button.
Version control is not an area that can be partially implemented or neglected. It is essential to have a clear versioning system for the artifacts going into the assembly line in order to assure that matching parts are fused together and the process can be reliably repeated while the changes occurring between executions are well organized and documented. Mature implementations of Continuous Delivery require all configuration elements such as third party components, as well as web server and operating system settings to be stored in version control together with the application code, tests and dependencies. It is recommended to store every possible artifact needed to build and provision the entire environment together with configurations that can potentially change over time. Although standardizing on a consistent application platform and a robust application packaging solution can ease this requirement, it is still mandatory to store at least the complete code base and artifacts needed to build and package the application.
Continuous Integration
Continuous Integration (CI) is a methodology that evolved from extreme programming techniques based on the idea that developed applications should always remain in a working state and each break should be immediately fixed. With Continuous Integration, applications are built either at specific intervals, or in more conservative implementations, on every change checked in by the developers. The goal of Continuous Integration is to eliminate the traditionally lengthy and painful system integration phase, which often consumes much of the time of developers, testers and release managers who must patch code components together while clearing issues sequentially. The risk of conflicts can be reduced through frequent integration of the entire code base, starting from an early phase. Although it is possible to implement deployment automation without continuously integrating and frequently building the application, it is clear that CI increases productivity of the various teams by minimizing the integration phases. In addition, Continuous Integration is the process to ensure that the application is always in a working state and ready to be deployed with an assured level of quality provided by automated testing. Continuous Integration mandates the strict use of version control and places its trust in the developers discipline to ensure that changes are well-tested locally and merged into the main source trunk as frequently as possible, and that unit testing is an integral part of the process. In a test-driven development environment, tests are developed before or together with the applications business logic. This increases the reliability of a Continuous Integration practice as it allows unit testing of each piece of functionality in an automated fashion. Zend views the Continuous Integration process as a combination of three distinct phases - Build, Automated Testing and Packaging - executed sequentially with validation steps for added control over the deliverables and the quality of the manufactured application.
Commercial and open source CI solutions enable the orchestration of the CI pipeline, effectively managing and synchronizing the tasks required to build the application. Among the popular CI server choices in the PHP ecosystem are Jenkins, Hudson and Bamboo. They coordinate the Continuous Integration workflow and ensure an organized, repeatable build cycle rather than relying on manual scripts or a command line based process. The Continuous Integration server is integrated with the version control system and will automatically fetch source code and dependent artifacts to kick off a build cycle either on scheduled intervals (usually at least once per day), or as each new revision of a code file or other artifact is checked in to the main version control trunk.
Build
After pulling code and artifacts from the version control system, the Continuous Integration server launches the build process. For compiled languages, the build stage is more complex and follows a number of compilation steps, linkage, and building binaries or for some languages platform-specific executables. Most modern compilers feature the ability to compile and link code to executable binaries via command line, thus enabling automated builds. These are performed from either the CI server directly, or by calling a specialized build tool (generic or language-specific) such as Apache Ant, Gnu make, MSBuild or PHPs popular phing. PHP and similar dynamic languages benefit from a fairly straightforward build step since they do not require compilation and linkage. The result is fewer errors in the build stage. At the same time, more focus on automated testing helps to identify syntax
and runtime issues early. As a dynamic, interpreted language, PHP generates code suited to multiple platforms because it eliminates the need to manage multiple flavors of binaries.
Automated Testing
When implementing a CI practice, it is necessary to enforce automated testing, which addresses both quality assurance of new functionality and regression protection. Automated testing that targets individual code segments on every change ensures that each component entering the assembly line is validated to an acceptable degree. Developers must create unit tests for each component being developed and keep tests updated in sync with requirements and code changes. The depth of testing within the CI process is fairly diverse. At the minimum, organizations should execute their suite of unit tests after pulling the latest version of the code and prior to building the application. More advanced quality controls also add a layer of static code analysis to ensure proper code styling, adherence to best practices, and further assurance that the code is both efficient and maintainable. These validation tests secure the fundamental operation of the new build. After build and packaging, the application is deployed to the test server. Continuous Delivery calls for another set of system-wide automated tests including end-to-end transaction execution. Such testing confirms the quality of the integrated assembled system as a whole and the success of the packaging stage as a result. Functional test and application-wide static code analysis allow the assembly line to provide a wider safety net for scenarios that go undetected by unit tests, and identify potential future issues resulting from poor coding. Organizations equipped to implement sophisticated test automation practices can run more advanced tests, simulating the production environment to test security, scalability and load, at the end of the CI phase. Because such testing at the CI phase often requires provisioning of complex testing environments, it is typically performed in a dedicated acceptance testing phase after the application is packaged and a proper acceptance testing or staging infrastructure has been provisioned. Basic tests for load and endurance scenarios during the CI phase do not replace the need for subsequent validation in a production-like or staging environment prior to pushing the application to production.
The most modern and comprehensive approach for packaging and release automation uses a dedicated packaging and deployment solution that is either infrastructure-agnostic or provided by the application platform. Use of such packaging and deployment solutions eliminates the dependency on specific infrastructures and their packaging formats, as well as infrastructure automation tools. The portable nature of these formats allows packaging to become an integral phase of the CI process. It is a common best practice to mimic the configuration of the expected production system as the default across testing and staging environments. However, business requirements may dictate a choice of production environments, which creates the need to either: (1). override a certain configuration when deploying on target production environments, or (2). support multiple production environments in parallel (e.g. diverse clouds and hybrid on-premise and cloud environments). In these scenarios, a comprehensive infrastructure-agnostic packaging solution alleviates the need to address different configurations at build time. The packaging solution can deliver a self-contained deployable format supporting deploy-time configuration adjustments based on parameters unique to the target environment type. The configuration of a specific production environment is then refined by the package installer, which can also enforce dependencies. Deploy-time configuration is crucial to automated deployment, extending flexibility to apps that rely on external dependencies and complex system configurations. An example of this scenario is third party dependencies such as payment gateways or central storage, which are not identical between testing and production. Deployment -time configuration scripts are commonly used in database and web server configurations or upgrades, and they allow the same package to be deployed on different web servers (i.e. Apache and Nginx), or different database setups (e.g. cloud database platforms). An application platform such as Zend Server, with its release automation and application monitoring capabilities, can be used to automate the process of creating and executing these scripts. This reduces the risk of configuration inconsistency across environments. Functional testing of the application should be performed until stable configurations are established across the multiple expected environments while application monitoring practices are implemented to detect errors in all actual target production environments. The Continuous Integration system pulls code from a version control repository, performs the builds (including dependencies), and creates an application package using the packaging system interfaces. Automating all steps, from fetching the code to providing a deployable package and performing them in every integration cycle, reduces the risk of a failed delivery. Following successful application packaging, the best practice is to deploy the application on a test server and immediately perform consistency and basic functionality checks to ensure the quality and functionality of the packaged application. This step, often referred to as a smoke test, validates application functionality in production. The complete, tested portable package is stored in the CI server or checked in the version control system to ensure consistency. Additional deployments of this specific build in testing, staging or target production environments should be performed using the same package. Just as it is ill-advised to dismantle and reassemble a vehicle once it moves off a manufacturing assembly line, the same principle holds true for an application once it has been tested and packaged. A release automation system that provides portable, reusable packaging followed by distribution to provisioned environments dramatically increases the efficiency of the Continuous Delivery process and gives businesses the flexibility to take advantage of diverse production targets. Continuous Delivery aims to keep software in a production-ready state so a package will be ready for deployment at any time to any environment. Use of release automation technologies is not as widespread as one might expect, but adoption is growing steadily. According to the 2012 Zend Developer Pulse survey [3], about 42% of developers at companies with more than 1000 employees and 38% in smaller companies practice some form of application deployment automation. Adoption of a packaging and release automation solution, whether a simple script-based solution or preferably a comprehensive, platform managed solution, is essential to achieve the production-ready state with each software build that constitutes a true Continuous Delivery practice.
Another key element in a mature Continuous Delivery environment is the automated rollback mechanism. It serves as an insurance policy in cases of failed deployments in production. Several techniques can be used for rollback independently or combined. Traditional practice includes variations on building a parallel environment and swapping traffic at the load balancer, leaving the previous environment available for a certain period of time in case problems are detected and there is a need to revert back to the old environment. This practice is sometimes referred to as a blue-green deployment: the parallel blue production environment is ramped up and at the right moment switched to green meaning live production. Other techniques involve rolling upgrades where only part of the environment is upgraded at any given moment and the process can be stopped when problems are detected. Utilizing an application deployment management system that can deploy packages to production and support automatic rollback significantly reduces risk in the release process and promotes zero downtime deployments. Other benefits include suitability for very small software increments and lower cost of frequent releases. Rollback managed by a deployment system such as the one provided by Zend Server consumes fewer resources: deployments can be performed on the same machines while eliminating some of the potentially risky moving parts. Since updates and rollbacks may execute custom tasks (e.g. DB changes or data migrations), deployment and rollback should be thoroughly tested during acceptance testing.
Infrastructure Automation
With Continuous Delivery, two layers of automation are required to fully provision and deploy an application: Release Automation and Infrastructure Automation. The Infrastructure Automation system focuses on automating the provisioning of runtime environments or platforms required to run an application, and gives the ability to provision virtual machines (generic and specially configured images) for the desired application runtime environment. This is usually done by adding an application server or runtime platform, a database or other application infrastructure elements. Traditionally, provisioning environments has been a manual, time-consuming and error-prone task that eats up valuable IT organization resources. Repetitious manual configuration of databases, application servers and web servers is a complex task that tends to create future problems of inconsistent implementations. Infrastructure Automation eliminates the manual labor associated with configuration of network elements, load balancers, user management, web and application servers, and databases. Infrastructure Automation is fundamental to cloud management systems. When combined with virtualization management subsystems, it paves the way for programmatically managed, on-demand infrastructure (Infrastructure-as-a-Service or IaaS). Add to this provisioning capability the application server and other middleware layers and you have a Platform-as-a-Service (PaaS) solution whereby the hardware, OS and runtime environment are completely virtualized and consumed on demand. Solutions for Infrastructure Automation include distributed agents on the servers, a centralized workflow management and rules processing engine, and scripting language support for orchestration of the production environment. These solutions can eliminate costs associated with manual provisioning and release steps. With automation, environments can be provisioned and launched at will, or terminated and discarded when obsolete. The process is faster and more reliable because manual intervention steps have been eliminated. The programmatic nature of such solutions ensures that environments are identical and always provisioned in the same manner and configured in the same way.
10
increases as the level of automation reaches the application stack and its configuration. Therefore standardization on an automatically provisioned application platform provides a higher level of confidence than OS-level standardization. Maintaining a consistent application stack allows organizations to eliminate environmental issues early in the development cycle and accelerate the developers problem resolution capability by taking environment variables out of the equation. It also allows developers to build the applications and troubleshoot incidents with a better understanding of the target production system. Many unexpected behaviors can be avoided by mandating a controlled, consistent application stack or platform. The fact that many application platforms are designed with portability in mind and supported by major cloud vendors allows organizations to use consistent application runtime stacks and be free to choose their production environments based on cost optimizations rather than specific technical characteristics.
11
12
When Zend works with customers deploying the Zend Server application platform, we encourage their DevOps teams to leverage application monitoring and management capabilities early on. Starting in early development, and continuing throughout the app lifecycle including CI, automated test and User Acceptance Testing. In this way, developers not only can find problems earlier using more strict thresholds in pre-production and solidify the application, but also refine and set proper thresholds for monitoring rules packaged with the application and later deployed in production. Developers also become familiar with use of monitoring and root cause analysis tools early, well before critical issues find their way into the production environment. To support operations team requirements for centralized monitoring and improved problem detection and isolation, platforms such as Zend Server can forward application-level alerts to infrastructure management solutions such as Nagios and Tivoli. This gives operations staff greater visibility into the application environment so they can push intelligent escalations through the feedback loop.
13
14
Benefits Consistency
Zend Server provides the ability to ensure a consistent application stack and configurations across dev, test and production. This includes library management capabilities that ensure framework versions and other dependencies are correctly managed across environments. The configuration management capability of Zend Server ensures configuration consistency across clustered environments in dev, test and production, and alerts the team to prevent errors caused by configuration inconsistencies. This capability also enables an easy copy of settings from staging to production via an automated application deployment process.
Automation
With Zend Server, PHP applications can be automatically deployed from Continuous Integration into production through an automated packaging and release process. Zend Server provides a packaging and deployment capability for PHP applications that enables their automated deployment from dev to staging and from staging to production, which can be integrated with automated infrastructure deployment through automation software such as Chef or Puppet. It also provides automatic rollback capabilities that significantly reduce risk in the release process. Zend Server also enables automatic scaling and elasticity, by providing a capability to automatically provision identical configurations across a cluster or multiple cloud instances, and add fully-configured instances instantaneously when needed. Configurations can also be managed in an automated manner, where any missynchronizations (a common cause of production errors) are automatically corrected.
Collaboration
Zend Server equips dev and ops teams with shared visibility into an applications behavior, via a detailed set of application-level behavior diagnostics. These diagnostic and management capabilities include monitoring an applications performance and responsiveness, monitoring for critical errors, logging and managing changes to configurations, and providing code execution level visibility into application issues. User management capabilities allow easy access to information from both development and operations teams, but with different usage rights (developers cannot modify production settings but can see detailed diagnostic information).
15
RELEASE AUTOMATION
provides Automated Deployment Cluster Deployment Dependency Validation Automated Rollback Auto-deploy on Scale-up Library Versioning
INFRASTRUCTURE AUTOMATION
provides Automated Provisioning Automated Configuration Control Elastic Scaling
APPLICATION MANAGEMENT
provides Application Monitoring Performance Management Root Cause Analysis Change Tracking Shared Dev/Ops Visibility
Learn more, and see the latest list of Zend Patterns at www.zend.com/blueprint.
Summary
Businesses need agility to respond to ever-changing market demands and produce apps rapidly and with quality. Weve seen how agile development supports faster innovation. And yet the dynamic and unpredictable nature of modern production environments has created the need to pair agile development with agile application delivery faster software releases synchronized with business needs on a consistent basis. Continuous Delivery allows organizations to streamline and automate traditional app delivery processes that are manual, error-prone and costly in nature. The software manufacturing process is being transformed into an automated assembly line able to continuously iterate and deliver new code to production. Organizations are embracing DevOps and adopting Continuous Delivery to attain the goal of business agility and increased collaboration. We hope this paper has been helpful in outlining the steps and approach to implementing a professional Continuous Delivery platformone that solidifies the IT organizations position as a vital corporate asset in increasing the competitiveness of the company and quickly addressing the needs of the business.
16
About Zend
Zend partners with businesses to rapidly deliver modern apps across web, mobile and cloud. We helped establish the PHP language, which today powers over 200 million applications and web sites. Our flagship offering, Zend Server, is the leading Application Platform for developing, deploying and managing business-critical applications in PHP. More than 40,000 companies rely on Zend solutions, including NYSE Euronext, BNP Paribas, Bell Helicopter, France Telecom and other leading brands worldwide.
Corporate Headquarters: Zend Technologies, Inc. 19200 Stevens Creek Blvd. Cupertino, CA 95014, USA Tel 1-408-253-8800 Fax 1-408-253-8801 Central Europe: (Germany, Austria, Switzerland) Zend Technologies GmbH, St.-Martin-Str. 53, 81669 Munich, Germany Tel +49-89-516199-0 Fax +49-89-516199-20 International: Zend Technologies Ltd. 12 Abba Hillel Street, Ramat Gan, Israel 52506 Tel 972-3-753-9500 Fax 972-3-613-9671 France: Zend Technologies SARL, 105 rue Anatole France, 92300 Levallois-Perret, France Tel +33-1-4855-0200 Fax +33-1-4812-3132 Italy: Zend Technologies, Largo Richini 6, 20122 Milano, Italy Tel +39-02-5821-5832 Fax +39-02-5821-5400 Ireland: Zend Technologies, The Digital Court, Rainsford Street, Dublin 8, Ireland Tel +353-1-6908019 2013 Zend Corporation. Zend and Zend Server are registered trademarks of Zend Technologies Ltd. All other trademarks are the property of their respective owners.
0106-M-WP-1013-R1-EN 17
www.zend.com
Zend Technologies, Inc. www.zend.com