It seems to me to be almost miraculous when you commit something and a couple of minutes later it’s working in the dev environment, having been tested already, and with no effort by us.
Continuous Integration and Continuous Deployment (CI/CD) are very well known software practices, and in this blog I will share the experience of our team in implementing the CI/CD pipeline.
We have multiple micro services and a couple of web apps that make up our application. Thinking of manually deploying all these would have been a nightmare considering we have a slim product development team at useMango.
When any new service is introduced in our product we make sure that the CI/CD pipelines are implemented for that service from day 1. We plan our sprints accordingly. Another practice that we follow is that the team member who implements/changes the service is responsible for creating/maintaining the CI/CD pipeline for that service.
Our velocity initially took a hit when we started this process, as every team member had to learn how to implement CI/CD. However later it paid off as any new successful commit would be automatically be deployed to our dev environment and available for everyone to consume. Any new bugs are identified and fixed faster. There is no dependency on a single DevOps expert as all the team members are equally equipped to fix issues in the CI/CD pipeline.
All our project repositories have corresponding Continuous Integration pipeline for both the dev and the master branches. The CI pipelines check that the project builds successfully and that the unit tests for that project pass after each new commit.
A new build is triggered on a Continuous Integration pipeline after every commit to the related branch. We use Azure DevOps pipelines for our CI pipelines.
Benefits of having a CI pipeline are well known: smaller code changes are more atomic and have fewer unintended consequences. Important aspect here is to have good coverage of unit tests which are run by the CI pipeline.
We use YAML for writing our CI pipeline. YAML has great benefits over the earlier approach of using the classic builder for building CI pipelines. Since you commit the YAML definitions directly to source control, you get all the benefits of a source control system. Some of the important benefits to note are the ability to see the version history, review differences between two commits, rollback changes in case of any issues, etc.
The CD pipelines are responsible for continuous deployment of the services to the cloud. We use Azure DevOps Releases for our CD pipelines. Deployment pipelines are linked to CI pipelines and a deployment is triggered after every successful build.
Azure DevOps allows creating deployment pipeline by adding steps. There are various predefined steps available for all types of deployment. If you are deploying a Lambda function, Than you will need to add a AWS CLI step to generate a cloud formation template and a AWS CloudFormation Create/Update stack step to update the stack on AWS.
Integration testing is an important aspect of software development and has many benefits that we all know. Our CD pipelines run integration tests automatically after each service is deployed.
Having a good coverage of integration tests is important. Integration tests allows early detection of errors at the interface level of the software. This builds confidence in the quality of the deployed software.
Our Team relies very heavily on the Unit Tests and the Integration tests to test the quality of our software.
We do deployment in three stages: Dev, QA and Live. The dev branch deploys through the dev stage to a dev environment. The master branch deploys to the QA and Live stages/environments successively.
Dev deployments are for use by the development team, making it easy for developers to test their changes.
Once a feature or fix looks ready, we merge to the master branch, triggering the QA deployment stage. The QA stage supports additional automated and manual testing prior to final release.
Azure DevOps allows stages to be triggered by manual approval and we use this for the live stage. This provides the final gate for releases to the live environment when the team believes that the service is complete and correct for use by customers.
While building your pipeline you may need to specify multiple configuration data. Some of the configuration data may be confidential information like a database connection string. Security is an important aspect of software development. It is bad practice to have your confidential information stored in the CD pipeline or in a YAML file.
We choose to use the System Managers Parameter store in AWS for this purpose. Some of the benefits of using the parameter store are that it allows to securely store all the configuration data, the configuration data is now on the cloud and can be shared across multiple pipelines, configuration data is stored in hierarchies and is version tracked. Last but not the least, AWS toolkit for Azure DevOps provides ready to use tasks to read parameter from the Parameter store.
Speed is the name of the game and it is important to be able to increase the speed to market of IT solution. Regardless of whether your team has CI/CD in place or not, when your team can see and feel the difference in their development speed, your executives will be thankful that you bit the bullet to bring about hard but necessary changes for the company. With all this to gain, if your team hasn’t started implementing CI/CD yet, what are you waiting for?