What is Change Failure Rate?
Change Failure Rate (CFR) is an extremely helpful metric that identifies the percentage of workflows that fail to enter production and the overall risk that this poses to development.
Change Failure Rate takes all your workflows over a period and calculates the percentage that ended in failure or required remediation (e.g., require a hotfix, rollback, fix forward, patch).
Change Failure Rate is a DORA metric and a core DevOps metric, and more broadly a core Agile delivery metric. There are four DORA metrics popularised by the DevOps Research and Assessments (DORA) group: the others are Deployment Frequency (DF), Lead Time for Changes and Mean Time to Restore Service (MTTR).
Example Change Failure Rate chart – Plandek DORA metrics dashboard
The calculation of Change Failure Rate requires organisations to surface data from CI/CD tools (e.g. Jenkins, CircleCI). This is done via an analytics plug-in or via an end-to-end delivery metrics dashboard like Plandek’s LiveView.
Example Change Failure Rate drill-down chart – Plandek DORA metrics dashboard
Plandek’s powerful filtering enables Change Failure Rate analysis by a number of different dimensions – such as branch, pipeline, portfolio, programme, repository and team – in order to reduce it.
Key use cases
Deployment failures are a key source of friction in the end-to-end delivery process and waste time and resources. With that in mind, Change Failure Rate is a very useful DevOps metric that helps teams reduce their overall Lead Time and increase the velocity of software delivery.
Change Failure Rate is particularly important for organisations that intend to increase delivery velocity and reduce Lead Time. This often includes:
- delivery organisations at an early stage of Agile DevOps maturity.
- organisations with large-scale delivery capabilities who utilise distributed teams (onshore, offshore, contractor, inhouse) and may have high engineer turnover.
- teams involved in strategically-critical software delivery projects.
Reducing Change Failure Rate will reduce overall Lead Time and increase software delivery velocity and quality.
Google and the DevOps Research & Assessment group publish a study each year based on a worldwide survey of DevOps professionals. Teams are tiered depending on their relative performance.
Below is the Change Failure Rate breakdown from the 2019 survey.
|Tier||Change Failure Rate||% of Teams in Tier|
As you can see, a reduction in Change Failure Rate is easier to achieve when an organisation has increased their Agile DevOps maturity. This is where Change Failure Rate can support an organisation regarding both immediate projects as well as the health of the organisation’s Value Stream Management.
Plandek is a global leader in software delivery metrics and analytics, recognised by Gartner as a top nine global vendor in their DevOps Value Stream Management Market Guide (published in Sept 2020).
Plandek works by mining data from toolsets used by delivery teams (such as Jira, Git, CI/CD tools and Slack), to provide end-to-end delivery metrics & analytics to optimise software delivery predictability, risk management and process improvement.
Plandek is based in London and works with clients globally to apply predictive data analytics and machine learning to deliver software more effectively.