Insight-driven Agile Delivery© at a time of remote working, uncertainty and cost pressure

The world has changed very dramatically, and a “new normal” has appeared almost overnight – a time of remote working, great uncertainty, changing priorities and dramatic cost pressures.  Software delivery teams sit at the heart of this challenging new environment as organisations look to them to deliver more for less in strategically critical areas.

Metrics, visibility, governance, and risk management were already an increasing priority in Agile software delivery – particularly in large-scale organisations. But recent events have catapulted them from “important” to “critical”.

Our experience shows that it is not feasible to deliver software effectively in this new normal environment without a complete end-to-end view of the software delivery process across all teams.

The Plandek BI and analytics tool draws data from the multiple tool sets that underpin the software delivery process to surface metrics and analytics to continually improve the delivery process.  We call this powerful methodology “Insight-driven Agile Delivery” or IDAD.

This short whitepaper discusses the principles of IDAD and how it can be applied today to deliver software efficiently in the new normal world of remote working teams placed under great pressure.

The case for a data-driven approach to software delivery in the new normal world

Software delivery is a complex, people-driven process with many interdependent stages and contributors.  In addition, software delivery teams often do not respond well to rapid change and uncertainty.

Hence, the new normal world of enforced remote working, rapidly changing priorities and pressure on cost and resources are likely to negatively impact delivery performance.  Visibility and metrics, therefore, become increasingly critical to identifying problems early and mitigating risk.

Happily, the software delivery lifecycle (SDLC) is underpinned by a huge amount of data residing in the toolsets used across the SDLC.  Key toolsets in this regard include:

• Workflow management tools (like Jira, Rally, AzureDevOps)
• Code repositories
• Code quality tools
• CI/CD tools
• Service management tools

It is the very large data footprint that resides in these tools that can provide actionable insight to underpin more effective software delivery at times of change and stress (see Figure 1 below).

The software delivery lifecycle (SDLC) is underpinned by a huge amount of data residing in the toolsets used across the SDLC.

Insight-driven Agile Delivery© seeks to leverage these data to continuously improve delivery effectiveness and reduce delivery risk.

The principle of Insight-driven Agile Delivery

It is our assumption that in adopting an Agile software delivery approach, organisations are trying to deliver software in keeping with the principles of the original Agile Manifesto.

The first of the twelve Agile Principles is to increase customer satisfaction through “the early and continuous delivery of valuable software”.  All the other principles (and values) support this core commitment.

IDAD looks to surface a set of metrics that track the success of this core Agile commitment – and present the insight to all levels within the delivery organisation – from leadership to team level, which is especially critical when team members are working remotely and are under increased pressure to deliver.

Critically also, IDAD looks to surface leading metrics that are deterministic of improving the process (and predicting likely outcomes) – rather than lagging metrics that simply “look in the rear-view mirror”.

The 5 core metrics that underpin IDAD and which are critical in the new normal world

We are not trying to reinvent the wheel with IDAD – we are simply offering a set of leading metrics that track your ability to continuously improve against the central Agile principle of software delivery.

These metrics are shown in Figure 2 below.  They are well-known “Agile metrics” and form the basis of the IDAD methodology.  They can be viewed at an aggregate level by the technology leadership, and as we will see in section 5, they are supported by a set of sub-metrics that directly drive these core metrics.

These sub-metrics are cascaded down the organisation to the team level so that teams work to continuously improve them and hence deliver to the technology leader’s overall delivery goals.

The fifth metric (see Figure 2 below) tracks individuals’ feelings of engagement/morale, which are particularly important at a time of remote working and great uncertainty, such as when we find ourselves in this new normal world.

Figure 2. High-level governance metrics that underpin IDAD

Cascading IDAD metrics – our top 5 metrics for managers and teams in the new normal world

The power of metrics can only be realised if they are vocally sponsored by leadership and are then cascaded down the organisation to the key management areas and Agile teams (squads) responsible for software delivery.

Typically, the management areas most directly involved in the end-to-end software delivery process include Engineering, Quality Engineering, Delivery, Programme Management, and DevOps.

As shown in Figure 3, the cascade of metrics falls into all these areas.

Figure 3. Cascaded IDAD metrics – management and team levels

In the new normal world, some of these metrics become particularly important.  Our top 5 are:

1. Deployment Frequency. Critical DevOps metrics such as Deployment Frequency which confirm our ability to maintain the continuity of our delivery.
2. Flow Efficiency. Development efficiency metrics such as Flow Efficiency (the % of time tickets spend in an active versus inactive status) are likely to be negatively impacted as teams move to unfamiliar remote working.
3. Cycle Time and Lead Time. Cycle Time and Lead Time – the critical measures of time to value, which are also likely to suffer in times of change and stress.
4. Completion Rate. Completion rate, which measures our ability to deliver our sprint goals over time.   This is highly likely to be negatively impacted in the new normal world and will then adversely affect the predictability of delivery timing.
5. Engineer Morale Score. Can be measured in Plandek via polling using collaboration tools like Slack.  This is a critical measure in the new normal world as people adjust to the often unfamiliar set of circumstances that they find themselves in.

Integrating IDAD with your OKR or GQM framework – flexibility to reflect your particular objectives

IDAD is not intended as a rigid framework.  Instead, it is intended as the basis from which clients can build their own bespoke metrics sets that more closely mirror their specific objectives in this new normal environment.

IDAD is not intended as a rigid framework.  Instead, it is intended as the basis from which clients can build their own bespoke metrics sets that more closely mirror their specific objectives in this new normal environment.

Many organisations will take the IDAD metrics listed in this paper as a good place to start.  We would agree, as the IDAD metrics ensure that you continue to deliver against the most central Agile principles at a time of rapid change and stress.

However, there are a variety of commonly used metrics setting approaches – such as OKRs (Objectives and Key Results) or GQMs (Goal, Metric, Question) as popularised by Victor Basili, which can also be applied to define a bespoke metric set.

In our view, it is the discipline of tracking and managing metrics (cascaded throughout the delivery organisation) that is critical in this new normal world.