Blog

Effective governance and risk management using end-to-end metrics in times of rapid change

Plandek, January 14, 2020

Software development is a people process.  Quality outcomes are dependent on motivated and well organised teams.  As a result, periods of rapid change (e.g. a post-merger integration or period of cost reduction) can precipitate serious delivery risk, as engineering teams struggle to work within the constraints imposed upon them. 

As a result, during periods of change and stress, it is vital to have a clear understanding of software Delivery Capability Risk (DCR) at team level, to ensure that delivery velocity and quality is maintained – and delivery risk is successfully identified and mitigated. 

The start point – the need for effective governance and risk management increases in times of change/stress

At Plandek, we work across a great variety of software delivery environments – from start-ups, to enterprises – from high growth teams, to teams undergoing periods of retrenchment and cost reduction.

All are increasingly interested in governance and risk management of their end-to-end software delivery process, to try and increase the four fundamentals of:

  1. Creating great working environments with highly engaged teams
  2. Reducing time to value
  3. Improving quality; and
  4. Increasing timing accuracy (predictability).

Effective governance and risk management is underpinned by quantitative analysis of a set of end-to-end delivery metrics that are determinant of delivery risk.  These metrics require collating data from the underlying toolsets used across the SDLC (such as workflow management tools, code repositories, CI/CD tools and system management tools).

And as delivery risk rises naturally at times of stress, experience shows that this sort of governance and risk management is especially important at times of change/increased cost pressure.

Understanding Delivery Capability Risk in times of heightened delivery risk

Understanding software delivery risk in totality is a complex task, with a range of internal and external factors at play.

Arguably the most critical risk factor is Delivery Capability Risk (DCR) that is made up of the delivery risk within the software delivery team itself.  And it is this risk that is most affected by increased stress caused by periods of intense change/cost reduction.

The concept of DCR is summarised in the graphic below.  There is a great range of Enterprise Agile Planning solutions that help you manage your delivery journeys (programmes).  They track scope, effort and apparent progress.  What they are unable to do is “lift the bonnet” on your delivery capability to understand your DCR – the very real risk sitting across the teams responsible for design, development, testing, builds and deployment.

 

Graphic showing the concept of Delivery Capability Risk Assessment

Figure 1 – Graphic showing the concept of Delivery Capability Risk Assessment

 

Plandek integrates with planning tools to surface a set of metrics that accurately track delivery capability risk (DCR) in times of increased stress/change.

It creates a balanced set of metrics that determine delivery capability risk, using both quant data from the underlying tools sets such as Jira, Git etc – and also from the engineers themselves via constant polling through Slack or other collaboration hubs – which is especially important during times of stress induced by rapid change or cost pressure.

The metrics fall into five logical categories which when synthesised together, give an accurate measure of DCR when tracked over time.  These categories are:

  1. Backlog health analysis – metrics and analytics to understand as far as is possible the state of team’s backlog, especially as it relates to the current and next programme cycle;
  2. Talent – quant metrics to understand your delivery teams’ morale and views on process effectiveness (collected via polling on collaboration hubs);
  3. Process efficiency and transparency – metrics that reveal the effectiveness of the end-to-end delivery process (e.g. Flow Efficiency and Lead Time analysis) which reveal bottlenecks and friction in the process;
  4. Throughput and time to value – metrics showing volume of work being produced and time taken to deliver across the end-to-end SDLC.

Delivery (sprint) accuracy – metrics showing teams’ ability to meet their own sprint goals (for Scrum Agile) which is a key determinant of the likelihood of delivering over longer time periods (e.g. Programme Increments).

Examples of these metrics are shown in Figure 2 below.

Example end-to-end software delivery metrics that determine Delivery Capability Risk (DCR)

Figure 2 – Example end-to-end software delivery metrics that determine Delivery Capability Risk (DCR)

 

This balanced scorecard of capability risk metrics adds a critical dimension to programme management at times of heightened delivery risk.

The importance of real-time feedback from teams in times of increased delivery risk

Plandek enables technology leadership to poll individual engineers on an ongoing basis via collaboration hubs like Slack and Microsoft Teams.  Because Plandek is integrated with the workflow, different questions can be asked at different points in the process – e.g. on the receipt of a Ticket, completion of a Ticket, completion of a Sprint etc.

We call this live survey capability “Team Pulse” and it is especially valuable at times of rapid change and stress.  Because there is a great body of research (and common sense) which shows that individual motivation and engagement is critical in successful software delivery outcomes.  As such team members view of:

  1. How effectively they feel the process/team to be working; and
  2. How they are feeling

…are both key determinants of delivery risk.

And our research shows that (1) and (2) are among the first things to suffer at times of rapid change or top down pressure (due to cost reduction for example).

So, as shown in the example in Figure 3 below, Plandek’s Team Pulse functionality provides technology leadership with a continuous picture of how individual engineers, QA managers etc are feeling, as part of the overall management of delivery capability risk.

 

Typical Plandek Team Pulse dashboard showing aggregated talent satisfaction data (score out of 5)

Figure 3 – Typical Plandek Team Pulse dashboard showing aggregated talent satisfaction data (score out of 5)

 

The importance of applying delivery capability risk to project management frameworks at times of stress/change

Our experience shows that effective programme planning and management is particularly difficult at times of change/stress.

Planning tools can show the theoretical progress of projects relative to planned milestones or Program Increments (in scaled Agile environments).  However, what these tools cannot do, is effectively track the health of the underlying technology delivery capability – and how that is influencing the observed progress to plan.

For example, a programme planning report may show that we just hit a key milestone, but an understanding of the health of the underlying delivery team may paint a very different picture – especially when teams are under increased stress.  It may show that this was achieved in an unsustainable way (low morale, declining process efficiency, increasing technical debt etc) – hence the team is unlikely to hit the next milestone.

This is why an understanding of delivery capability risk (i.e. understanding the health of the underlying delivery “engine”) is a vital extra dimension – especially at times of rapid change or increased stress.

And it is why, even in times of tight cost control (when expenditure on tooling may be frozen or reduced), Plandek is used as a delivery risk management tool to be applied in conjunction with existing Enterprise Agile Planning tools (such as Jira, Jira Align, Rally etc).

The three Plandek case studies overleaf illustrate examples of how Plandek is applied to reduce delivery risk and improve productivity at times of rapid change and stress.   

 

 

 

 

Introduction to Plandek

Plandek (www.plandek.com) is the leading Agile metrics BI and Analytics platform to help large Agile technology teams better manage delivery risk and improve their effectiveness.

Plandek works by mining the data history from dev teams’ toolsets (e.g. Jira, Git) to reveal and track metrics that are highly predictive of improved Agile project outcomes.  These metrics are often not visible or accurate with current toolsets.  Plandek is working with a wide range of organisations, to track and actively manage these Agile metrics, to manage software delivery risk and improve performance.

Read more by Charlie Ponsonby:

You can also check out Charlie’s article on the importance of metrics to Agile teams on InfoQ, which recently trended globally.

Read more about our team.