There’s an old adage – “what gets measured gets done”. But metrics are a contentious subject in Agile software delivery, with some Agile practitioners contending that Agile metrics are a bad thing, as at best they can be gamed and manipulated by teams – and at worst may instil a negative culture of “Big Brother is watching you” detrimental to Agile team wellbeing.
However, this negative view of Agile metrics is very much on the decline – with most more mature Agile DevOps organisations recognising the core role that Agile metrics play if carefully selected and therefore adopted enthusiastically by Agile teams.
So what are the Top 5 Agile metrics that engage both teams and managers alike and avoid the pitfalls outlined above?
Criteria for choosing our Top 5 Agile metrics
We have chosen the following criteria to select our Top 5 Agile metrics:
- Firstly and most importantly, the Agile metrics chosen track the core underlying objective of Agile software delivery – “the early and continuous delivery of valuable software” (Agile Manifesto Principle Number 1), https://agilemanifesto.org/iso/en/principles.html
- second, the metrics selected are valuable to teams and managers alike – indeed it is vital that teams adopt Agile metrics to drive their own Agile continuous improvement (as per Principle 12 in the Manifesto)
- the metrics are meaningful when considered at team level and when aggregated across teams
- and finally, the Agile metrics are simple to understand and if tracked, very quickly drive significant improvement in Agile software delivery effectiveness.
Our Top 5 Agile metrics
In no particular order, here are our Top 5 Agile metrics that make an immediate impact on your Agile software delivery.
Cycle Time is a basic measure of your organisation’s Agility, in that it measures your velocity, or the time taken to develop an increment of software. Unlike the more comprehensive Agile metric of Lead Time (which measures the length of the entire end-to-end delivery process), Cycle Time is easier to measure as it looks only at the time taken (within a scrum team) to take a ticket from the backlog, code and test that ticket – in preparation for integration and deployment to live.
As per Figure 2 below, the Cycle Time metric view allows teams to understand time spent in each ticket status within the development cycle. Analytics tools that offer filtering enable analysis by Status, Issue Type, or Epic (and any other standard or custom ticket field) all plotted over any time range required.
Figure 2 – Example Cycle Time metric view
Deployment Frequency is another fundamental measure of an organisation’s agility (when viewed alongside the other critical metrics described here).
A core objective of Agile delivery is the ability to develop and deploy live small software increments rapidly. Deployment Frequency is an Agile metric that tracks that base competence and is a powerful Agile metric around which to focus effort at all levels in the delivery organisation at the early stages of an Agile transformation.
Figure 3 – Example Deployment Frequency metric view
Delivered Story Points
Delivered Story Points is often considered a problematic Agile metric due to the potential inconsistencies in the calculation of story points and how much effort they represent. However, as a basic measure of output and how that is changing over time, it is a powerful Agile metric around which to align.
There may be concerns of teams ‘gaming’ the metric with story point inflation, but as with all Agile metrics, they should be viewed in context by experienced folks who know the teams well. And if this is the case, they can still give an excellent view of how the delivery organisation is progressing over time.
Figure 4 – Example Delivered Story Points metric view
Escaped Defects is a simple but effective Agile metric of overall software delivery quality. It can be tracked in a number of ways, but most involve tracking defects by criticality/priority as per the example below.
Figure 5 – Example Escaped Defects metric view
When these four simple Agile delivery metrics are viewed together, the Agile DevOps practitioner can get a good balanced view of how their Agile DevOps maturity is progressing.
Importantly, the Agile metrics can be tracked over time, making sure that an improvement in one metric (e.g. Cycle Time) does not lead to a detrimental effect on another metric (e.g. Escaped Defects).
In addition, the relationship between Cycle Time and Deployment Frequency can be closely watched. Very often teams are able to reduce their Cycle Time, but this does not translate into quicker value delivery, due to bottlenecks in the integration and deployment process.
Our final Agile metric in our Top 5 is Flow Efficiency. Flow Efficiency looks at the proportion of time tickets spend in an ‘active’ versus ‘inactive’ status) and is a great Agile metric for teams.
The Flow Efficiency analysis (see Figure 7 below), enables Team Leads to isolate and analyse each ‘inactive’ status in the workflow and consider if there is scope to reduce or eliminate it. The analysis shows the relative size of each ‘inactive’ status opportunity in terms of time spent in the inactive state and volume of tickets affected.
Figure 7 – Example Flow Efficiency metric
Typical opportunities to remove inactive bottlenecks include time spent with tickets awaiting definition (e.g. Sizing) and tickets awaiting QA. Where waits for QA are considered excessive, Delivery Managers can reconsider QA resource allocation by team.
About the author
Charlie Ponsonby is co-founder and Co-CEO of Plandek based in London.
Plandek was recently recognised as a top nine global vendor by Gartner in the fast-growing ‘DevOps Value Stream Management Platforms’ space and is used by enterprise clients globally to improve the effectiveness of their software delivery capability.
Plandek provides a cloud-based analytics platform to help software delivery teams deliver quality software, faster and more predictably. It works by mining data from delivery teams’ toolsets (such as issue tracking, code repos and CI/CD tools), to provide actionable and intelligent insight (including Agile metrics) across the end-to-end software delivery process.