What is Build Failure Rate?
Build Failure Rate is a commonly used Agile DevOps metric. As the name suggests, Build Failure Rate helps you understand how frequently your builds are failing and, in particular, which pipelines experience the most failures, allowing you to focus the team’s attention in the right areas.
As such, Build Failure Rate is an important measure of Agile DevOps maturity: build failures pose an overall risk to smooth software delivery and are a common problem in teams adopting a CI/CD (continuous integration/continuous deployment) methodology.
Build Failure Rate helps track and manage a significant source of risk both in day-to-day development and responding to incidents due to delays. Observing all your workflows over a period of time, it calculates the percentage that ended in failure.
It is similar to Change Failure Rate which focuses on deployment failures and is one of the four DORA metrics popularised in Forsgren, Humble and Kim’s popular book, Accelerate: The Science of Lean Software and Devops: Building and Scaling High Performing Technology Organizations.
Example Build Failure Rate metric view – Plandek DevOps metrics dashboard
As one of the metrics in our Top 5 DevOps metrics guide, Build Failure Rate is a key DevOps KPI. Alongside Build Failure Rate, Mean Build Time, Mean Failed Build Time and Mean Time to Recover from Build Failures are also good measures of how much time is being wasted on failed builds, as well as how effectively DevOps teams can recover from failed builds when they occur.
Build Failure Rate is also often viewed alongside the DORA metrics: Deployment Frequency (the frequency at which new releases deploy to production), Lead Time For Changes (the time between commit to production), Mean Time to Restore and Change Failure Rate.
Key Use Cases
While other engineering metrics – such as Time to Recover from Build Failures – show how quickly your teams are responding to failure, Build Failure Rate gives an indication of the overall health of a pipeline.
It’s important to remember that Build Failure Rate does not track how badly a build has failed, but simply whether or not it has failed. Despite the binary pass/fail categorisation, the overall failure rate is a handy DevOps metric to gauge overall risk in development.
As with all DevOps metrics, engineering metrics and Agile metrics, it is key to look at Build Failure Rate over the relevant time period, and by limiting the history of builds in your Build Failure Rate calculation you can be sure it remains relevant. Build Failure Rate over 30 days, for instance, can provide sufficient information for a snapshot of project health in many organisations, although Plandek’s dashboards can give a longer or shorter-term view as needed.
Occasional build failures are acceptable as long as they are fixed quickly, although high Build Failure Rates suggest product instability, which could be attributed to any number of issues, including design flaws, tricky bugs, or pipeline faults. As such, Build Failure Rate may not shine a light on the specific root cause of the failure, but will show up where the failure rate requires further investigation.
Research has shown that a Build Failure Rate of 20% is average, while 30% or more indicates the need for improvement.
Plandek is a global leader in software delivery metrics and analytics, recognised by Gartner as a top nine global vendor in their DevOps Value Stream Management Market Guide (published in Sept 2020).
Plandek works by mining data from toolsets used by delivery teams (such as Jira, Git, CI/CD tools and Slack), to provide end-to-end delivery metrics & analytics to optimise software delivery predictability, risk management and process improvement.
Plandek is based in London and works with clients globally to apply predictive data analytics and machine learning to deliver software more effectively.