Agile DevOps metrics are an increasingly hot topic. Up until recently DevOps metrics (sometimes known as DevOps KPIs) have been constrained by the analytical capabilities of the underlying DevOps toolsets.
But with a new generation of Value Stream Management tools, it is possible to surface and synthesise data from multiple DevOps tools (e.g. workflow management tools, code repos, CI/CD tools and APM/ITSM tools). As a result, a new wave of more powerful DevOps metrics and DevOps analytics are now helping DevOps practitioners to deliver valuable software much more effectively.
Selecting the top 5 DevOps Metrics
We have selected the most popular DevOps metrics among our clients and have focused on those DevOps metrics that give you ‘the biggest bang for your buck’ whatever level of Agile DevOps maturity you are currently at.
The DevOps metrics selected are similar to the DORA metrics as popularised by the DevOps Research and Assessments (DORA) group. But we have based our top 5 DevOps metrics selection based on actual feedback (and usage) from a wide variety of Plandek clients at all levels of Agile DevOps maturity. As such, our Top 5 DevOps metrics reflects what we actually see being used to drive value among our clients!
Our Top 5 DevOps metrics
In no particular order, here are our Top 5 DevOps metrics that make an immediate impact on your software delivery.
Deployment Frequency is perhaps the most fundamental DevOps metric as it measures the ability of an organisation to develop and deploy to live small software increments rapidly.
More specifically it tracks the frequency with which increments of code are deployed to staging, testing and production.
As the core objective of Agile software delivery is ‘..the early and continuous delivery of valuable software..’, Deployment Frequency is a core Agile metric and represents a core objective of effective DevOps. Deployment Frequency is also one of the four DORA metrics.
Example Deployment Frequency chart – Plandek DevOps dashboard
The calculation of Deployment Frequency requires surfacing data from CI/CD tools (e.g. Jenkins, CircleCI). This is typically done via reporting plug-ins to such tools or via an end-to-end delivery metrics dashboard like Plandek (www.plandek.com). Delivery metric dashboards like Plandek enable you to analyse delivery, engineering and DevOps metrics in a number of different ways. For example, you can analyse the frequency of deployments by branch, portfolio, programme, repository or team. As such targets can be set and practical actions taken to increase deployment frequency.
Example Deployment Frequency drill-down chart – Plandek DevOps dashboard
Code Cycle Time
Code Cycle Time is a key DevOps metric as it tracks the efficiency of the Pull Request process – which itself is a critical element of the end-to-end software delivery process.
More specifically, Code Cycle Time analyses all completed Pull Requests (PRs) (e.g. closed, merged, declined etc) within the specified time range and shows the average hours to complete, from when the PR was opened. Not only that but Code Cycle Time provides full insight into the different stages that a PR goes through (e.g. Time to Review, Time to Approve, Time to Merge/Close and Time to Deploy)
The method of calculation of Code Cycle Time is shown below:
Example Code Cycle Time chart – Plandek DevOps metrics dashboard
A powerful DevOps analytics tools like Plandek enables you to analyse Code Cycle Time by a number of different filters in order to identify bottlenecks and reduce it significantly. Such filters include analysis by PR author, PR participant, Code Cycle Time Stage, Respository and Ticket Issue Type.
Example Code Cycle Time drill-down chart – Plandek DevOps metrics dashboard
Mean Build Time
Mean Build Time is a very impactful DevOps metric for organisations at all levels of Agile DevOps maturity. It analyses the time taken for a build and is typically analysed by workflow.
It is a very good indicator of core DevOps process health as a steadily increasing mean workflow build time will drive longer Cycle Times.
We particularly like filtering by status to help you keep an eye on slow builds which ultimately end in failure.
Example Mean Build Time view by Workflow Name
Mean Time to Recover from Build Failures
Mean Time to Recover from Build Failures is a key DevOps KPI as build failure is such a common occurrence and the main source of friction in the deployment process. As such it needs to be tracked and managed.
The metric identifies how long it takes for the next successful workflow on a branch that has failed. Mature Agile DevOps organisations become very good at managing this recovery time and so improve their deployment frequency.
Example Mean Time to Recover from Build Failures metric view
Flakiest Files is a really helpful DevOps metric aimed to identify fragile source code files in your codebase which can be targeted for refactoring in order to reduce your Build Failure Rate.
The darker the shade of red the more often a commit has resulted in a failed build. Plandek’s drill-down enables you to filter by file extension to help focus on a particular subset of files.
Example Flakiest Files DevOps metric view
Build Failure Rate
Eagle-eyed readers will have noticed that this is the sixth metric in our Top 5 DevOps metrics guide, but we’ve included it as it’s another key DevOps KPI that really shouldn’t be left out!
Indeed, Build Failure Rate is an extremely helpful DevOps metric as it identifies the percentage of workflows which fail and the overall risk this poses to development.
As such it helps track and manage a significant source of risk both in day to day development and responding to incidents due to the delays. Taking all your workflows over a period it calculates the percentage that ended in failure.
Example Build Failure Rate DevOps metric view
About the author
Charlie Ponsonby is co-founder and Co-CEO of Plandek (www.plandek.com) based in London.
Plandek was recently recognised as a top nine global vendor by Gartner in the fast-growing ‘DevOps Value Stream Management Platforms’ space and is used by enterprise clients globally to improve the effectiveness of their software delivery capability.
Plandek provides a cloud-based analytics platform to help software delivery teams deliver quality software, faster and more predictably. It works by mining data from delivery teams’ toolsets (such as issue tracking, code repos and CI/CD tools), to provide actionable and intelligent insight (including Value Stream metrics) across the end-to-end software delivery process.