Effective governance and risk management using end-to-end metrics in times of rapid change

Software development is a people process.  Quality outcomes are dependent on motivated and well-organised teams.  As a result, periods of rapid change (e.g. a post-merger integration or period of cost reduction) can precipitate serious delivery risk, as engineering teams struggle to work within the constraints imposed upon them. 

As a result, during periods of change and stress, it is vital to have a clear understanding of software Delivery Capability Risk (DCR) at the team level, to ensure that delivery velocity and quality are maintained – and delivery risk is successfully identified and mitigated. 

The start point – the need for effective governance and risk management increases in times of change/stress

At Plandek, we work across a great variety of software delivery environments – from start-ups to enterprises – from high-growth teams to teams undergoing periods of retrenchment and cost reduction.

All are increasingly interested in governance and risk management of their end-to-end software delivery process to try and increase the four fundamentals of:

1. Creating great working environments with highly engaged teams
2. Reducing time to value
3. Improving quality; and
4. Increasing timing accuracy (predictability).

Effective governance and risk management are underpinned by quantitative analysis of a set of end-to-end delivery metrics that are determinants of delivery risk.  These metrics require collating data from the underlying toolsets used across the SDLC (such as workflow management tools, code repositories, CI/CD tools and system management tools).

And as delivery risk rises naturally at times of stress, experience shows that this sort of governance and risk management is especially important at times of change/increased cost pressure.

Understanding delivery capability risk in times of heightened delivery risk

Understanding software delivery risk in totality is a complex task, with a range of internal and external factors at play.

Arguably the most critical risk factor is Delivery Capability Risk (DCR) which is made up of the delivery risk within the software delivery team itself.  And it is this risk that is most affected by increased stress caused by periods of intense change/cost reduction.

The concept of DCR is summarised in the graphic below.  There is a great range of Enterprise Agile Planning solutions that help you manage your delivery journeys (programmes).  They track scope, effort and apparent progress.  What they are unable to do is “lift the bonnet” on your delivery capability to understand your DCR – the very real risk sitting across the teams responsible for design, development, testing, builds and deployment.

Figure 1 – Graphic showing the concept of Delivery Capability Risk Assessment

Plandek integrates with planning tools to surface a set of metrics that accurately track delivery capability risk (DCR) in times of increased stress/change.

It creates a balanced set of metrics that determine delivery capability risk, using both quant data from the underlying tools sets such as Jira, Git, etc. – and also from the engineers themselves via constant polling through Slack or other collaboration hubs – which is especially important during times of stress induced by rapid change or cost pressure.

The metrics fall into five logical categories, which, when synthesised together, give an accurate measure of DCR when tracked over time.  These categories are:

1. Backlog health analysis – metrics and analytics to understand as far as possible the state of the team’s backlog, especially as it relates to the current and next programme cycle;
2. Talent–quant metrics to understand your delivery teams’ morale and views on process effectiveness (collected via polling on collaboration hubs);
3. Process efficiency and transparency – metrics that reveal the effectiveness of the end-to-end delivery process (e.g. Flow Efficiency and Lead Time analysis), which reveal bottlenecks and friction in the process;
4. Throughput and time to value – metrics showing the volume of work being produced and the time taken to deliver across the end-to-end SDLC.
5. Delivery (sprint) accuracy – metrics showing teams’ ability to meet their own sprint goals (for Scrum Agile), which is a key determinant of the likelihood of delivering over longer time periods (e.g. Programme Increments).

Examples of these metrics are shown in Figure 2 below.

Figure 2 – Example of end-to-end software delivery metrics that determine Delivery Capability Risk (DCR)

This balanced scorecard of capability risk metrics adds a critical dimension to programme management at times of heightened delivery risk.

The importance of real-time feedback from teams in times of increased delivery risk

Plandek enables technology leadership to poll individual engineers on an ongoing basis via collaboration hubs like Slack and Microsoft Teams.  Because Plandek is integrated with the workflow, different questions can be asked at different points in the process – e.g. on the receipt of a Ticket, completion of a Ticket, completion of a Sprint, etc.

We call this live survey capability “Team Pulse,” and it is especially valuable at times of rapid change and stress.  There is a great body of research (and common sense) that shows that individual motivation and engagement are critical in successful software delivery outcomes.  As such, team members’ views of:

1. How effectively do they feel the process/team is working, and
2. How they are feeling

…are both key determinants of delivery risk.

Our research shows that (1) and (2) are among the first things to suffer at times of rapid change or top-down pressure (due to cost reduction, for example).

So, as shown in the example in Figure 3 below, Plandek’s Team Pulse functionality provides technology leadership with a continuous picture of how individual engineers, QA managers, etc, are feeling as part of the overall management of delivery capability risk.

Figure 3 – Typical Plandek Team Pulse dashboard showing aggregated talent satisfaction data (score out of 5)

The importance of applying delivery capability risk to project management frameworks at times of stress/change

Our experience shows that effective programme planning and management is particularly difficult at times of change/stress.

Planning tools can show the theoretical progress of projects relative to planned milestones or Program Increments (in scaled Agile environments).  However, what these tools cannot do, is effectively track the health of the underlying technology delivery capability – and how that is influencing the observed progress to plan.

For example, a programme planning report may show that we just hit a key milestone, but an understanding of the health of the underlying delivery team may paint a very different picture – especially when teams are under increased stress.  It may show that this was achieved in an unsustainable way (low morale, declining process efficiency, increasing technical debt, etc) – hence, the team is unlikely to hit the next milestone.

This is why an understanding of delivery capability risk (i.e. understanding the health of the underlying delivery “engine”) is a vital extra dimension – especially at times of rapid change or increased stress.

And it is why, even in times of tight cost control (when expenditure on tooling may be frozen or reduced), Plandek is used as a delivery risk management tool to be applied in conjunction with existing Enterprise Agile Planning tools (such as Jira, Jira Align, Rally etc).

The three Plandek case studies below illustrate examples of how Plandek is applied to reduce delivery risk and improve productivity at times of rapid change and stress.   

About Plandek

Plandek (www.plandek.com) is the leading Agile metrics BI and Analytics platform to help large Agile technology teams better manage delivery risk and improve their effectiveness.

Plandek works by mining the data history from dev teams’ toolsets (e.g. Jira, Git) to reveal and track metrics that are highly predictive of improved Agile project outcomes.  These metrics are often not visible or accurate with current toolsets.  Plandek is working with a wide range of organisations to track and actively Agile metrics manage software delivery risk, and improve performance.