Reconnect with DevOps productivity best practices

From the builder to the building block of enterprise success

For the longest time, developers were just expected to build things. They were given a set of tools, told what to build, and seen as a cost centre. As companies have become totally software-driven, that perspective has shifted. Instead of seeing the development team as this cost centre to control, top enterprises are thinking about how to maximise the capabilities and productivity of these expensive, hard-to-hire, skilled experts. In fact software delivery is a huge competitive differentiator. Continuous integration and deployment have become key in achieving success for businesses, and so the decision-making power of developers has grown.

Continuous deployment has become the aspirational goal for most software delivery teams - and so they remain in demand at all times. For the always-on team, it’s important to stay fresh and keep striving to hit the top of their game.

This year has been tough. The minute COVID-19 really impacted society, enterprises realised that they could not rely on access to a build machine sitting underneath a developer’s desk. Being able to move quickly and reliably has become core to what software delivery teams have to do as first-class individuals with budget and decision power to keep the whole organisation delivering.

Develop better practices by learning from industry

CircleCI has massive amounts of unique data on how technology delivery teams perform. This year’s 2020 State of Software Delivery: Data-Backed Benchmarks for Engineering Teams report examines over 55 million data points from more than 44,000 organizations and 160,000 projects. The software delivery patterns observed, especially the data points from top delivery teams, show key similarities that suggest valuable benchmarks for teams to use best practices.

• Throughput: The number of workflow runs matters less than being at a deploy-ready state most or all of the time.
• Duration: Teams want to aim for workflow durations in the range of five to ten minutes.
• Mean Time to Recovery: Teams should aim to recover from any failed runs by fixing or reverting in under an hour.
• Success Rate: Success rates above 90% should be your standard for the default branch of an application.

And beyond these goals, some best practices, from the big to the niche, are important to keep up as we head into what we hope will be a better year.

From the technical...

Know when code Is being deployed: If new versions of services are being deployed throughout the day, make deployments visible to the wider team. You can use various solutions, like CircleCI, Rollbar, and Slack together to make sure everyone knows when code is being deployed.

Prevent broken code from being deployed: Some bugs do make it past tests, and are deployed to production. One class of bugs that often slips through unit tests are where (well-tested) software components are configured or combined incorrectly right at the top level of the app. If a team uses Kubernetes, for example, they can use its power to give an extra safety net when deploying, to catch such bugs. Consider a service running on three pods. When deploying a new release of the service, one pod is started running the new code. Once this pod is healthy, one of the old pods is terminated, and a second new pod is started. Rolling the pods one-by-one ensures there are three healthy pods in production as code deploys. If a new pod fails to start, Kubernetes will restart the new pod, leaving the remaining pods in place.

Ensure messages are delivered: Any service that runs in production might be terminated at any time, and the danger is that a service might consume a message from a message queue, and be shut down before it has a chance to take the required action. This has the same effect as the message being lost. Ensure teams can defer acknowledging the receipt of messages until they have performed the required action.

… To the corporate

Think ‘fast’: The value of speed is clear. For IT organisations it’s no longer about control - it’s about building tools on behalf of all the software development teams inside of an enterprise and helping everyone collaborate.

Think long-term: The way you scale is by making the mundane highly leverageable. For example, can everyone on the team act as a great recruiter? Does the office space (or remote working practices) encourage open collaboration, ideas sharing, and simple handovers? Are tools helping more than holding back? Look at every part of the system and optimise for scalable and repeatable success.

Keep learning: DevOps teams are doing incredibly interesting things: from building autonomous vehicles to space vehicles to machine learning to everyday mobile applications, and everything in between. Stay humble and keep learning from the industry. Find your heroes and follow them - keep learning from success stories.

Hire dedicated engineering managers: Often you’ll have engineers managing engineers, but management is a separate skill-set. And as technology execs transition through growth stages and rise within the business, they need to hire great technical leaders who can be given autonomy.

DevOps has a bright future in 2021, and reconnecting with the right mind-set will set the team up to succeed. Whether a coder or a manager - ensure the team secures headspace to focus on upskilling, being productive, and focusing on the factors that drive scaleable success. For those businesses scaling up from seed to series C, it’s worth taking a look at the Sifted ‘Building tech teams’ report, made with CircleCI, to learn from those who have successfully managed their transformations.