What is First Time Pass Rate?
First Time Pass Rate (%) measures the percentage of Tickets that pass QA the first time without stimulating a return transition or defect sub-task. As such, this metric is an excellent measure of overall team health.
Too often this metric is seen as an engineering quality metric when it is actually a better reflection of how well a team is working together and supporting one another.
A First Time Pass requires the inter-dependent elements of an Agile development team to be working well by showing:
- strong collaboration and communication between team members throughout the delivery process;
- well-defined Tickets that are clearly understood by developers and QA testers;
- reduced WIP levels, allowing engineers to focus on a single Ticket at one time;
- a well-supported QA process that’s engaged throughout the process and clear on the functionality that needs to be tested.
When all of these things are in place, First Time Pass Rate can be optimised. As such, it’s a good overall Agile software delivery KPI.
First Time Pass Rate is often considered in tandem with Return Rate (%), a similar quality metric. The exact definition of both these software delivery metrics is shown below:
As shown in the example First Time Pass Rate summary chart below, First Time Pass Rate can easily be tracked over time as part of an ongoing continuous improvement initiative.
It is a powerful software delivery metric as it is meaningful when aggregated (e.g. at tribe, programme, and release train level). It is also critical at the delivery team level and should become a core Agile metric for review in team retrospectives and planning meetings.
A software delivery and engineering metrics platform like Plandek enables you to drill down into a metric like First Time Pass Rate. You’re able to see trends over time, the Tickets themselves that regressed, and variations in the metric at key stages in the delivery process (e.g. QA).
First Time Pass Rate, when considered at the individual engineer level, should never be used as a performance review metric – rather, it is a critical team metric as it is influenced by so many factors beyond the individual engineer’s control.
Related metrics
First Time Pass Rate is often considered with the similar and related quality metric Return Rate. Return Rate (%) tracks the overall percentage of Tickets returned from QA. This may include Tickets that have been returned multiple times.
As discussed above, First Time Pass Rate is most illuminating when viewed in tandem with other delivery metrics relating to the effectiveness of the team – as it is so affected by interdependencies within the team. Other relevant team metrics include:
- Measures of backlog health such as Story Points Ready For Development.
- WIP (the number of work items started but not completed at any one time).
- Sprint Completion Rate – a measure of the overall dependability of the team and its ability to hit its Sprint goals.
Key use cases
First Time Pass Rate is typically used as an Agile delivery quality metric, particularly at the team level. As described, it is an excellent measure of the overall team and process health and acts like the ‘canary in the coal mine’ in that it is a good leading indicator of underlying issues in team and process health.
As such, it is often used within a broader balanced scorecard of Agile delivery and engineering metrics to track Agile DevOps maturity.
Expected outcomes
By improving this metric, you will see an immediate improvement in throughput – AKA reduced Cycle Time – and a likely indirect positive impact on Sprint completion and team morale.
A First Time Pass Rate of over 90% is achievable by mature Agile DevOps practitioners and high-performing teams.
Conversely, teams with a low average First Time Pass Rate will struggle to deliver effectively – they will have longer Cycle Times, larger QA resource requirements and frustrated engineers.
About Plandek
Plandek is an intelligent analytics platform that helps software engineering teams deliver value faster and more predictably.
Celebrated by Gartner and Forrester as a ‘leading global vendor’, Plandek mines data from delivery teams’ toolsets and gives them the opportunity to optimise their delivery process using both intelligent insights and predictive analytics.
Co-founded in 2017 by Dan Lee (founder of Globrix) and Charlie Ponsonby (founder of Simplifydigital), Plandek is based in London and currently services the UK, Europe, the Middle East and North America.
Find out more about Plandek here: The Plandek Difference.
