Value stream metrics to drive software delivery productivity
The increased cost of capital and riskier economic environment in 2023 is changing the way organisations think about technology investment.
As belts are tightened, CTOs are under pressure to do more with less and to provide clear evidence that their software delivery teams are delivering value as efficiently and predictably as possible.
As such, it is no surprise that Value Stream Management is seen as ‘the next big thing’ in DevOps. The concept is simple (and borrowed from lean management principles) in that it encourages software delivery teams to accelerate the delivery of value by identifying and mapping their key value streams (which in Agile terms often relate to products), and then using metrics and related process improvements to optimise the efficiency of these value streams and accelerate the delivery of value to customers.
Metrics-led technology delivery – the size of the prize
Experience at Plandek across private and public organisations globally, is that very significant improvement in delivery productivity is possible in a relatively short period of time (2-6 months).
Examples include:
- A 400% improvement in Deployment Frequency of new features
- A 55% reduction in Time to Value (the time taken to deliver an increment of software)
- An 80% improvement in Sprint accuracy – the key determinant of delivering new features in a predictable way
- A 54% reduction in Escaped Defects to ensure that software quality actually improves whilst velocity is increased
- A 20% improvement in Throughput relative to time spent.
These are substantive improvements in delivery effectiveness rather than marginal increases and they are achievable by any organisation willing to focus sustained effort on increasing delivery effectiveness by becoming metrics-led.
Key success factors in adopting a metrics-led approach
We have observed five characteristics of organisations that have achieved very significant improvement through data-led software delivery:
- The sponsorship and drive of technology leadership
- Organisation around products or value streams
- The recognition that this is NOT about measuring individuals in a ‘Big Brother is watching you’ way – instead, it is about measuring and optimising an end-to-end PROCESS (the software delivery lifecyle), from design all the way through to delivery to live
- The selection of a ‘balanced scorecard’ (to coin a management cliché) of software delivery metrics to optimise the process and around which the organisation can align
- The provision of intelligent insight and customised dashboards for Product Managers and Team Leads to empower them to drive the continuous improvement required across the SDLC to realise the improved outcomes.
Indeed, it is our firm belief that the metrics-led approach must be ‘loved by teams and relied on by managers’ in order to succeed.
A top-down approach that does not bring the whole organisation with it will not succeed in driving improvement and may result in the unintended consequences of alienated engineering teams and distrusted/ignored metrics.
Key metrics to drive software delivery productivity
When selecting the ‘balanced scorecard’ of metrics around which to align (which we call ‘North Star’ metrics), it is helpful to think about answering two critical questions related to efficient value delivery:
- Is our technology team focused on our highest priority/value-creating initiatives? i.e. is it aligned to our strategic goals/roadmap?
- And if so, is our technology team delivering as efficiently as possible?
Strategic alignment metrics
Delivery analytics tools like Plandek are designed to enable you to easily answer question (1) above. Tech leadership can start to consistently track key strategic alignment metrics such as:
- Proportion of time, resource and cost expended, relative to key identified value streams or product areas and how this trending over time – i.e. are we expending effort on the things that really matter?
- The balance between effort expended on technical debt and bug fixing, versus building new features
- The proportion of backlog accounted for by different value streams/strategic priorities and how this is trending over time – to see if we are keeping on top of our roadmap in critical areas
- The relationship between value delivered and complexity (story points) delivered, and how this in turn relates to quality (defects). So that leadership can ensure that effort is not wasted on complex tasks which do not deliver value efficiently and/or generate quality issues.
In addition to considering whether our delivery capability is aligned around our strategic priorities (which is essential to maximise delivery ROI) – we can also consider the second question posed above – namely whether our technology team is delivering as effectively as possible?
Delivery productivity metrics
The example ‘North Star’ metrics dashboard below, shows a typical set of ‘North Star’ delivery capability metrics that objectively assess overall Agile DevOps maturity and delivery effectiveness.
As such, they represent operational KPIs that are relatable to the delivery and engineering team and which ultimately determine delivery productivity. Example metrics include:
Value Delivered
Value Delivered (R&D versus other). This is a core measure of value output for those organisations measuring the value of R&D effort, which is a fundamental measure of an organisation’s ability to deliver software and how that is changing over time.
Plandek enables companies to understand where R&D resource is being allocated – to better understand the proportion of tech effort/value delivered on vital roadmap work (R&D) – versus unplanned work, infrastructure improvement and technical debt (for example).
Lead Time to Value
Lead Time is a core agile software delivery metric which tracks an organisation’s ability to delivery software early and often, and provides a solid foundation upon which to assess where further investment will realise value most efficiently.
The concept of Lead Time is borrowed from lean manufacturing and captures overall time to deliver an increment of software from initial idea through to deployment to live – i.e. the complete end-to-end software delivery life cycle (SDLC). As such it is probably the first metric that the C-Suite should ask for to better understand how effectively a technology team is delivering.
The shorter the Lead Time, the higher the ‘velocity’ of the delivery team and tighter the feedback looks, hence the quicker the organisation is going to receive new features and respond to customer needs. Again, this is a vital KPI when assessing technology delivery efficiency.
Deployment Frequency
Deployment Frequency is another fundamental measure of Agile software delivery (or Agile DevOps maturity). A core objective of Agile delivery is the ability to develop and deploy to live small software increments rapidly. Deployment Frequency tracks that base competence and is a powerful metric around which to focus effort at all levels in the delivery organisation. Hence it is another key KPI for a ‘balanced scorecard’ of technology delivery capability.
Escaped Defects
Escaped Defects is a simple but effective measure of overall software delivery quality. It can be tracked in a number of ways, but most involve tracking defects by criticality/priority.
Any analysis of delivery efficiency should include consideration of Escaped Defect rates as it is clearly undesirable to nominally increase velocity and ‘productivity’ whilst reducing quality.
Throughput
Throughput can be measured in a number of ways (e.g. story points or tickets) and is the best proxy measure of ‘output’ in the broadest sense. This can then be compared relative to input metrics such as: time, headcount (of various types) and input costs (e.g. staffing cost).
Many engineering teams are reluctant to measure Throughput relative to proxy measures of input, but at a time of cost pressure (or at a time when investment is increasing over time) it is a critical measure to consider.
Sprint Target Completion
‘Scrum Teams’ (also known as squads) and ‘Sprints’ are the basic building blocks of Scrum Agile software delivery. If Scrum Teams consistently deliver their Sprint goals (a ‘Sprint’ typically involving a two-week increment of work), Agile software delivery becomes relatively predictable.
On the other hand, if Scrum teams fail to deliver their planned sprint goals, then it becomes impossible to predict delivery outcomes across multiple teams and longer time periods. Scrum team predictability (often referred to as ‘dependability’) is therefore a critical success criterion in Agile software delivery and should be assessed as part of a delivery productivity review.
High-performing Scrum teams will consistently have Sprint Target Completion rates in excess of 85%.
The above are some examples of relevant metrics to consider when tracking delivery productivity. The key is to take a balanced set of metrics that consider software delivery holistically as a complex process. There are many other metrics that are commonly used (and can be surfaced through Plandek) to assess a technology delivery capability, including the DORA metrics and Flow metrics.
Final thoughts on adopting metrics to drive delivery efficiency
When the pressure is on, there is a natural tendency to de-prioritise improvement initiatives in order to focus on the fire fighting. Our experience shows that this is the most likely way to perpetuate or exacerbate the current issues.
The investment of time and effort in implementing a data-led approach to improved delivery efficiency is never regretted. It can reap rewards within a quarter.
It’s similar to never getting round to making that new hire that you don’t have time to interview for. It’s a false economy to put it into the ‘tomorrow pile’!
About Plandek
Plandek is an intelligent analytics platform to help software delivery teams deliver valuable software, faster and more predictably.
Plandek enables technology teams to track and drive their improvement and share understandable KPIs with stakeholders interested in accelerating value creation/improving delivery efficiency.
As such Plandek is a key global vendor in the fastest growing area of DevOps known as Value Stream Management.
Plandek works by mining data from delivery teams’ toolsets (such as issue tracking, code repos and CI/CD tools), to provide actionable and intelligent insight across the end-to-end software delivery process for users throughout the delivery team – from Team Lead to the CIO.
Plandek is recognised as a top global vendor by Gartner and Forrester and is used by private and public organisations globally to optimise their technology delivery.
For enquiries please contact Sam Kange: skange@plandek.com
