Elements of DevOps

A bit of history

In the past, many fields of engineering have experienced a notorious evolution in the understanding of their own work. In software development, it isn’t different: in the last 3 decades, many IT organizations have evolved from waterfall to agile, and more recently, to DevOps.

The history of DevOps starts at the 2008 Agile conference in Toronto, Canada, where Patrick Debois and Andrew Shafer opened a session named “birds of a feather” on applying Agile principles to infrastructure. One year later, at the 2009 Velocity conference, John Allspaw and Paul Hammond presented their profound “10 Deploys per Day: Dev and Ops Cooperation at Flickr”, where they showed the audience how they were able to create shared goals between Dev and Ops and use continuous integration practices to include deployments as part of their daily routine. On the same year, Patrick Debois created the first DevOpsDays in Ghent, Belgium. Patrick was the first to coin the term “DevOps” too.

So, we can say that the DevOps methodology is the result of applying the most solid principles of physical manufacturing engineering and leadership to move IT organizations towards a culture of high-trust, speed and quality on software development. DevOps relies on a series of knowledge such as Lean, Theory of Constraints, the Toyota Production System, safety culture, resilience engineering, server as code, and many others.

The core, chronic conflict

In our world, many times Development and IT Operations are adversaries. Dr. Eliyahu M. Goldratt, a well-known name in the manufacturing management movement, called it “the core, chronic conflict”.

The main factor that contributes to creating this conflict is that both groups have competing goals. Development takes responsibility for responding to changes in the market and installing these changes into production as quick as possible. IT operations are responsible for providing stable, reliable and secure IT services to customers, making it hard to introduce production changes that can cause a bad impact to these customers.

Everything becomes just a little more difficult when applications and infrastructure are too complex, poorly documented, incredibly fragile, making technical debt ever-bigger. We promise to work on it in the future, but that time never comes. Ultimately, it creates a culture of fear, slowing deliveries, where feelings of powerlessness can easily emerge, where people become unwilling or simply unable to act in a way that avoids the same problems in the future.

A classic picture. Photo: Mandic (2019).

Exploring DevOps Toolkit

In DevOps methodology, a series of techniques can be found to alleviate or even resolve this conflict, and here I’ll let in on you some of them:

– By the usage of continuous integration, teams can assure their changes won’t break existing functionalities; and by the usage of continuous delivery, Development can automate production deployments to happen many times a day, making errors almost pass unnoticed to the customers as they become tiny small and easy to detect and fix.

– By the usage of Lean practices, the team can generate telemetry about the average lead time and C/A % (correct and accurate %) for each team, detecting the bottlenecks along the value stream, also reducing the number of times one requirement or defect goes back and forward.

– By the Lean philosophy of considering the internal customer (who receives the work immediately after us), developers are encouraged to see Ops as their most important customers.

– By the usage of dark launches and feature flags, IT organizations can separate for good the concepts of “deployment” and “release”, delegating the control and responsibility of the first to Development only and the control of the second to the Business, allowing the Business to release functionalities to customers independent of Development, according to the market time and investments strategies. Here I quote the remarkable words of Chuck Rossi, Director of Release Engineering at Facebook: “All the code supporting every feature we’re planning to launch over the next six months has already been deployed onto our production servers. All we need to do is turn it on.”

– By the usage of canary release pattern, Development gives to Business a powerful tool to release new functionalities for small audiences before rolling that out to larger audiences, without causing catastrophic issues in production.

– By the usage of strangler application pattern, Development receives a powerful tool to wrap all legacy code into microservices and APIs, controlling the number of requests they receive and giving to Ops team the control to gradually move these requests to the new code.

– By the usage of monitoring and production telemetry, an IT organization demonstrates to their customers that they don’t have anything to hide about the operation and performance of their systems, and, mostly, that they are prepared to catch and fix issues before they become hardships to the customers.

Additional materials

There are very good books about DevOps in the market. The book The DevOps Handbook – How to Create World-Class Agility, Reliability, and Security in Technology Organizations by Gene Kim is one of my favorites. For those who like novels and prefer soft readings, I recommend the book The Phoenix Project by Gene Kim too. The book Effective DevOps by Jennifer Davis is also very good.

For those who prefer interactive material like audio and videos, I suggest the courses DevOps: The Big Picture and Implementing DevOps in the Real World at Pluralsight. Richard Seroter has done a tremendous work compiling and putting all the various pieces of DevOps into play in these courses.

About the author

Rafael Melo is a Software Engineer at Poatek. He loves learning new things about software development, DevOps and other techniques to create a culture of high-trust inside companies. His hobbies include reading books of classical philosophy, history, politics and literature, going to the cinema, and more recently, playing with his arduinos.