The journey to becoming Agile can sometimes be tricky. This article discovers nine critical success factors that make Agile metrics work for teams. What questions should you and your team be asking yourself in order to focus on self-improvement, reliability, efficiency, and high-quality code delivery?

The article was originally published on JAXenter.

Inevitably, most teams get to the stage where they need to adopt a more Agile delivery process. This is not just a sign of maturity. It’s a sign that the software they are developing is being used, is deemed useful, and is receiving feedback and change requests so that it continues to improve.

My team is in a unique position. We strive to improve delivery as we develop a tool that strives to help teams do the same. In other words, we use our own tools to improve the delivery of it!

We are striving to improve delivery as we develop a tool that strives to help teams do the same. In other words, we use our own tools to improve the delivery of it!

In my experience, the journey to becoming more Agile can be tricky. Each team has its own goals and ideas about how to get there. However, all teams need to reflect on their progress, measure how effective their current strategy is, and gain more visibility of the wider landscape. Of course, this is easier said than done.

Bottom-up is best

The topic of which metrics Agile teams can trust to help them measure progress reliably – or whether to use them at all – is both fascinating and contentious. Many people associate metrics with a top-down management style, which is the opposite of the decentralised, empowered and self-determining team philosophy that Agile promotes.

During a one-to-one meeting with my team lead, I asked him which metrics he felt I should focus on. He explained that the only ones worth looking at were those that the whole team agreed would help improve delivery. When it came to my own self-improvement goals, he said I should select metrics myself.

As a rule, whenever metrics are applied from the top down, they are less effective. (This is not to say, however, that there aren’t valuable metrics that can indicate progress at a higher level.)

Using Agile metrics for team improvement

Self-improvement is a key Agile principle. On the face of it, it’s a pretty simple process. First, you identify what you want to improve. Next, establish ways to measure the attributes that contribute to improvement. Then, measure and reflect. Therefore, you will always need a reliable way to track progress.

First, you identify what you want to improve. Next, establish ways to measure the attributes that contribute to improvement. Then, measure and reflect. Therefore, you will always need a reliable way to track progress.

My Agile metrics that focus on various attributes of delivery, quality and value. For example, we measure Lead Time from when the ticket is created in Jira to its production deployment and the number of escaped bugs. We’ve created a dashboard in our own software around these attributes so we can measure, integrate and affect them daily or as part of a retrospective. Our dashboards help guide us and qualify decisions we make around team, process, and delivery improvement to ensure we continually head in the right direction. We can also opt to see individual contributions to these metrics. For instance, I have chosen to create a view of metrics that only I can see to measure my own output.

Using Agile metrics for delivery cycle improvement

As part of our cycle rituals, our team is responsible for ensuring our scope is realistic. To support this, we use Agile metrics to ensure that our tasks’ sum complexity, time and effort match the overall time available and the team’s abilities. We measure the scope using story points. We also built and now use a ‘Sprint Report’ facility. This allows us to see a breakdown of the sprint’s overall completion, including the target completion and work added to the sprint after it started. It also includes ‘Sprint-specific dashboards’ that use metrics like ‘Completed Tickets’ to calculate the amount of work developers can reliably complete during a sprint (aka their ‘velocity’).

As part of our cycle rituals, our team is responsible for ensuring our scope is realistic. To support this, we use Agile metrics to ensure that our tasks’ sum complexity, time and effort match the overall time available and the team’s abilities.

9 critical success factors that make Agile metrics work for teams

As I said before, Agile metrics for team improvement can be contentious. They open up a lot of heated discussions and, to varying degrees, benefit from a wider understanding of context and narrative. So, we discuss them and apply the following tenets to help find common ground:

Simplicity

Complicated metrics run counter to the Agile spirit. We like to define our journey as specifically as we can, answering simple questions with easy-to-understand metrics that support them, like:

• Are we effective at delivering value?
• What is our lead time?
• How long is our bug backlog?
• Are our sprints going well?
• Are we completing all tickets in a sprint?
• Are we adding lots of scope to sprints?
• Is team morale high?
• Does the team understand the business value of the work?
• Does the team feel autonomous?

Consensus

The development team must select the metrics and serve a common aim shared by project members from the Scrum Master to the technology leader.

Relevance

You shouldn’t measure anything unrelated to your journey’s destination. Each project follows a different set of milestones, so that it may need different metrics. However, as there is only one final destination, some carefully selected metrics should be applicable across all teams. Less can be more, so when we build out a dashboard together in our team meetings, we try to concentrate on only a handful of metrics at a time.

Significance

Software delivery metrics are often outcome-based. Although legitimate, tracking only symptoms and not root causes is risky. For example, the ‘Cycle Times’ metric shows how long work is taking rather than why. Descriptive metrics like these should also include details of the variables that impact the outcomes. For example, alongside Cycle Time, you could show an analysis of the bottlenecks. To improve, we want to uncover root causes and identify behaviour gains we can make together – we need to tell a full story.

Right sources – we need to analyse data from sources our developers genuinely engage with in their everyday work. These include workflow management software like Jira; code repositories like GitHub and Bitbucket, TFS or Gitlab; code quality tools like Sonarqube; time tracking systems like Harvest or Tempo; and continuous delivery tools like Jenkins and GoCD.

Automation

If analysing metrics takes significant cognitive effort or time to collate, we tend to lose patience and abandon the effort. The metrics need to complement processes, not slow them down.

Near real-time

Agile metrics delivered in near real-time fundamentally drive improvement, as they can be discussed in daily stand-ups and sprint retrospectives.

The human factor

Software development is a process (almost) completely driven by people. This means it should be possible to source information and quickly get to the root cause of issues. Typically, feedback is collected in person, in stand-ups and in retros. Theoretically, this should work well, but it can also hide issues that participants don’t want to communicate openly. This is especially true in changing, distributed teams with a mix of full-time employees and contractors. We incorporate feedback into our tool to address this and provide us with context and narrative around our metrics. For example, when tickets get closed, we can provide feedback on how the ticket went and its requirements via Slack. These prompts also give us a feel for how a ticket has performed post-dev as it continues (hopefully!) past QA.

Actionable

Metrics only make sense if teams can act upon and improve them. Without constant support, many teams will not have the time or inclination to drive their own self-improvement as they strive to meet their short-term delivery objectives. Active stewardship by the technology leadership team can make a huge difference.

The limits of home-grown solutions

Since many developers (Agile & metrics) don’t seem to get along, it’s no surprise that few analytics tools are available to measure Agile delivery effectiveness. However, now that Agile is mainstream, measuring is urgent. Since few analytics tools were available, many teams started to build their own. This worked well on a small scale but hit the wall when projects and teams grew.

There are several other problems with home-grown tools. Most notably, they allow teams to tweak calculations and tell an overly flattering story. Also, the time it takes to build your own tool can be a big distraction from planned work. Fortunately, new solutions are emerging that align with the principles listed above.

Agile metrics for self-improvement

If you are still unconvinced about using Agile metrics for teams, I recommend testing them on yourself. Most find that when they do this, the metrics become a reassurance or even a confidence boost. For example, a younger colleague of mine was struggling with his programming confidence. He found metrics to be very helpful because they showed him objective proof of improvement.

For my part, one way I often use Agile metrics is to provide insights during a retrospective. To measure how I improve over time, I track metrics for the tickets I’ve completed, the number of story points completed, and the number of returns I’ve had from QA. This also helps me remember the tickets I’ve worked on and how they went. Like most developers, I tend to switch focus once a ticket passes and can find it hard to retain the details when reviewing a cycle or performing a project post-mortem.

Whether you decide to use them yourself or for your team, (Agile & metrics) return true. In my experience, people want similar things and work well together in helping deliver on the key Agile principle of self-improvement.

You will, of course, come up with your own, but I have found these example questions (and related Agile metrics) can help self-improvement:

• Where has my time been spent? I find it interesting to look back on this. The time I spend on high-priority work drives velocity and productivity and helps me estimate delivery dates more confidently.
• How actively am I contributing towards ticket creation to improve quality?
• How much impact have I had? How much work did I get done, and what was it?
• Am I delivering high-quality code?
• How reliable am I?
• How efficient am I at delivering value?

Whether you decide to use them yourself or for your team, (Agile && metrics) return true. In my experience, people want similar things and work well together in helping deliver on the key Agile principle of self-improvement. Try it out!</p

About the author

Colin Eatherton is the Team Lead and Developer at Agile metrics software company Plandek. Colin develops the front end of Plandek’s UI using web technologies, including JavaScript, HTML, CSS and Ruby. Colin works with a cross-functional development team, stakeholders, and Plandek’s product manager in this role to understand and analyse delivery requirements and implement solutions. His team applies Agile methods and metrics to improve processes and outcomes continuously.