What is Build Failure Rate?
Build Failure Rate helps you understand how frequently your builds fail and, in particular, which pipelines experience the most failures, allowing you to focus the team’s attention on the right areas.
As such, this metric is an important measure of Agile DevOps maturity: build failures pose an overall risk to smooth software delivery and are a common problem in teams adopting a CI/CD (continuous integration/continuous deployment) methodology.
Build Failure Rate helps track and manage a significant source of risk both in day-to-day development and responding to incidents due to delays. Observing all your workflows over a period of time calculates the percentage that failed.
It is similar to Change Failure Rate, which focuses on deployment failures and is one of the four DORA metrics popularised in Forsgren, Humble and Kim’s popular book, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations.
Related metrics
As one of the metrics in our Top 5 DevOps metrics guide, Build Failure Rate is a key DevOps KPI. Alongside this metric, Mean Build Time, Mean Failed Build Time and Mean Time to Recover from Build Failures are also good measures of how much time is being wasted on failed builds, as well as how effectively DevOps teams can recover from failed builds when they occur.
This metric is also often viewed alongside the DORA metrics: Deployment Frequency (the frequency at which new releases deploy to production), Lead Time For Changes (the time between commit to production), Mean Time to Restore and Change Failure Rate.
Key use cases
While other engineering metrics – such as Time to Recover from Build Failures – show how quickly your teams are responding to failure, the Build Failure Rate indicates the overall health of a pipeline.
It’s important to remember that this metric does not track how badly a build has failed but simply whether or not it has failed. Despite the binary pass/fail categorisation, the overall failure rate is a handy DevOps metric to gauge overall risk in development.
As with all DevOps metrics, engineering and Agile metrics, it is key to look at the Build Failure Rate over the relevant period, and by limiting the history of builds in your Build Failure Rate calculation, you can be sure it remains relevant. Build Failure Rate over 30 days, for instance, can provide sufficient information for a snapshot of project health in many organisations. However, Plandek’s dashboards can give a longer or shorter-term view as needed.
Expected outcomes
Occasional build failures are acceptable as long as they are fixed quickly. However, high Build Failure Rates suggest product instability, which could be attributed to any number of issues, including design flaws, tricky bugs, or pipeline faults. As such, the Build Failure Rate may not shine a light on the specific root cause of the failure but will show up where the failure rate requires further investigation.
Research has shown that a Build Failure Rate of 20% is average, while 30% or more indicates the need for improvement.
About Plandek
Plandek is an intelligent analytics platform that enables software engineering teams to deliver value faster and more predictably.
Celebrated by Gartner and Forrester as a ‘leading global vendor’, Plandek mines data from delivery teams’ toolsets and allows them to optimise their delivery process using both intelligent insights and predictive analytics.
Co-founded in 2017 by Dan Lee (founder of Globrix) and Charlie Ponsonby (founder of Simplifydigital), Plandek is based in London and currently services the UK, Europe, the Middle East and North America.
Find out more about Plandek here: The Plandek Difference.
