Change Failure Rate - DORA Metrics

What is Change Failure Rate?

Change Failure Rate (CFR) is an extremely helpful metric that identifies the percentage of workflows that fail to enter production and the overall risk that this poses to development.

Change Failure Rate takes all your workflows over a period and calculates the percentage that ended in failure or required remediation (e.g., require a hotfix, rollback, fix forward, patch).

Change Failure Rate is a DORA metric, a core DevOps metric, and, more broadly, a core Agile delivery metric. There are four DORA metrics popularised by the DevOps Research and Assessments (DORA) group: the others are Deployment Frequency (DF), Lead Time for Changes and Mean Time to Restore (MTTR).

Example Change Failure Rate chart – Plandek DORA metrics dashboard

Calculating Change Failure Rate requires organisations to surface data from CI/CD tools (e.g. Jenkins, CircleCI). This is done via an analytics plug-in or an end-to-end delivery metrics dashboard like Plandek’s LiveView.

Example Change Failure Rate Chart Drilldown

Example Change Failure Rate drill-down chart – Plandek DORA metrics dashboard

Plandek’s powerful filtering enables Change Failure Rate analysis by a number of different dimensions – such as branch, pipeline, portfolio, programme, repository and team – in order to reduce it.

Key use cases

Deployment failures are a key source of friction in the end-to-end delivery process and waste time and resources. With that in mind, Change Failure Rate is a very useful DevOps metric that helps teams reduce their overall Lead Time and increase the velocity of software delivery.

Change Failure Rate is particularly important for organisations that intend to increase delivery velocity and reduce Lead Time. This often includes:

Delivery organisations at an early stage of Agile DevOps maturity.
Organisations with large-scale delivery capabilities which utilise distributed teams (onshore, offshore, contractor, in-house) and may have high engineer turnover.
Teams involved in strategically critical software delivery projects.

Expected outcomes

Reducing Change Failure Rate will reduce overall Lead Time and increase software delivery velocity and quality.

Google and the DevOps Research & Assessment group publish a yearly study based on a worldwide survey of DevOps professionals. Teams are tiered depending on their relative performance.

Below is the Change Failure Rate breakdown from the 2019 survey.

Tier	Change Failure Rate	% of Teams in Tier
Elite	0-15%	20%
High	0-15%	23%
Medium	0-15%	44%
Low	46-60%	12%

As you can see, reducing Change Failure Rate is easier to achieve when an organisation has increased their Agile DevOps maturity. This is where Change Failure Rate can support an organisation regarding both immediate projects as well as the health of the organisation’s Value Stream Management.