This software company situated in N America sells SaaS financial software to enterprise clients globally. The technology team of c250 people operates a scaled Agile framework. Teams are distributed geographically and are growing rapidly.
The experienced technology management team is committed to using end-to-end delivery metrics to empower engineering teams to self-improve over-time and to have a consolidated view across teams in order to share success.
Of particular interest is understanding how to really measure (previously identified) high-performing teams and therefore help other teams to apply the relevant metrics, to replicate success.
The scope of the Plandek metrics initiative was:
To find a balanced set of metrics that really characterise a ‘successful’ scrum team and to replicate the success across teams
As such, to encourage all teams to be data-led in their delivery and to create data-led “feedback cycles for teams to continuously get better”
To ensure that the metrics process is ‘bottom-up’ (team led) and not top-down (management imposed)
As such to create freedom for teams in selecting metrics to ensure a balance of “some prescriptive metrics” and freedom to choose at team level
To improve the Quarterly Review process by underpinning it with hard data and less anecdotal comment.
The Plandek Customer Success team worked closely with the client to track the data footprint of the identified ‘high performing teams’ to identify those metrics that best “explain” their success, in terms of delivery velocity, dependability and quality.
This metric set formed the basis of the ‘Shared Success’ metrics that the client socialised with all teams (via customisable Plandek team dashboards) to form the basis of their team level self-improvement process.
Cycle Time was found to be highly descriptive/deterministic of high performing teams. The high-performing teams all had Cycle Times for Stories <34% shorter than the average of all teams (working in comparable situations).
Flow Efficiency which examines the proportion of time tickets spend in an ‘active’ versus ‘inactive’ status was another key determining metric. High performing teams were found to have markedly higher rates of Flow Efficiency (>50% versus the team average of c30%)
Code Cycle Time a popular metric for DORA metrics fans (also known as Lead Time for Changes) was found to be highly correlated with high-performing teams.
First Time Pass Rate (FTPR) is another metric that was identified as deterministic of team success in that it is a key driver of Cycle Time improvement.
Story Points Ready for Development. The analysis showed that high performing teams had well managed backlogs (with at least 2 sprints worth of tickets prepared and ready to progress), enabling them to significantly reduce their Cycle Times.