Written by Filip Dimovski
In today’s fast-paced world of software development, it is necessary to deliver a product or a service promptly, while ensuring all the best security practices and considerations are taken into account and ensuring the infrastructure is capable of handling the requests of the modern web and mobile applications. These tasks need to be handled by several different engineers due to their complexity and the many areas they cover.
Development Operations (DevOps) as a software engineering concept involves a multidisciplinary approach, where software development, software and infrastructure security, and systems engineering and administration skills are used to streamline and improve the software development and delivery processes, as well as infrastructure management.
The core of the DevOps philosophy is the automation of processes, declaration of infrastructure resources, and reuse of components. Essentially, it sounds like software development, right?
Modern software development practice expects the use of software version control, such as GitHub, GitLab, Bitbucket, and others, which foster collaboration between software developers. It allows keeping versions of the code too, to ease auditing and resolution of software and security defects. The repository acts as a single source of truth, that must reflect the current state of software and infrastructure, and a place through which changes can be reviewed and applied in a controlled and automated manner.
One key area that speeds up the development of software and lets developers do more work is the automation of the lint, build, and test phases of changes introduced to a software component. This is accomplished by utilizing a CI/CD (Continuous Integration/Deployment) system, which consists of pipelines, declarations of steps that need to be executed to accomplish the aforementioned tasks. Such automation aims to find and resolve software issues quickly and efficiently and create reproducible builds of software artifacts, packaged software ready for deployment and execution in different infrastructure environments.
Docker containers are an excellent tool to easily prepare and ship applications as image artifacts that can be run on many different container orchestration platforms, such as Kubernetes. By packing the application and its dependencies (libraries, static content, etc.) in one container image, it can be delivered easily and executed on different environments, and a well-crafted container can be reused on several different environments, such as development, staging, and production. A developer can also run the whole application stack locally on their machine by using Docker Compose or Minikube, ensuring that the way their application runs is as similar as possible to production environment requirements.
Applications designed by employing the 12 Factors rules of software development and Docker containers can help achieve the universality and ease of deployment of applications. By leveraging environment variables for configuration, and keeping the all-state outside of containers and inside managed cloud databases, such as Amazon RDS, Elasticache for Redis, and similar, even if the application fails due to any factor, you can be sure that the data is preserved, and the application can be started back easily and quickly, due to the ephemeral stateless nature of containers. Also, many instances of the same application can be run in parallel on an orchestration platform, providing scalability when the usage is growing, such as when many users access the application at the same time.
Besides improving the speed of delivering software to production environments, one of the most important responsibilities is to ensure software, data, and infrastructure security. Together with Information Security (InfoSec) engineers, DevOps engineers can automate processes that check for security compliance, and adherence to standards, such as ISO 27001 and PCI, ensuring the software and infrastructure are secure for performing tasks related to monetary transactions, medical research, work with sensitive data, and similar.
If the software constitutes the main driving implementation of processes, the infrastructure is as important in ensuring the software works according to requirements. The DevOps way of managing infrastructure uses declarative formats, such as Terraform or CloudFormation templates, to define cloud resources and manage them in collaboration with other DevOps team members and version them in a code repository, and audit them using automated tools. The templates allow easy creation of new and modification of existing cloud infrastructure resources, and also constitute an excellent tool to ensure business continuity, in case disaster strikes and everything needs to be recreated from scratch and as soon as possible as part of disaster recovery and disaster contingency planning processes.
Lots of tools allow us to follow how the infrastructure resources and applications are performing, such as Prometheus, Grafana, the ELK stack, by collecting metrics and log data, allowing us to correlate them and discover points of slowdowns and contentions, discovering issues with the applications and infrastructure, and offer us a great tool for resolving such issues. Monitoring can be automated to inform teams when their applications do not perform as well as expected, and even allow the system to roll back in case of failures, increasing its resiliency and ensuring services are up and running for the clients.
One of the main concerns is data integrity. As most software represents a way to work with data, we must ensure that data is preserved in case of software errors, cyber-attacks, or disasters. That can be ensured by setting up automated replication of data in different datacenter regions, or even different services, snapshotting of stateful resources (databases, servers’ volumes), automated backups, and by defining a lifecycle policy of the data – how long it needs to be preserved and how. This can all be automated and performed consistently, by leveraging declarative infrastructure tools and writing scripts to perform automated backups according to schedule.
As we can see, many of these topics can be enhanced and many parts of the applications and the infrastructure can be improved by following design principles and processes defined by the DevOps software development philosophy. By leveraging the multidisciplinary approach and automating most of the processes, developers can ship software faster, and businesses can be more certain that things are going to perform well within the expectations.