Get started

Blog

Dependabot creating work… Understanding the impact of maintaining dependencies with Plandek metrics

Plandek, February 3, 2022

Key takeaways:

Using Plandek’s software delivery metrics platform to measure the number of pull requests created by an automated bot gave us a new insight into normally unseen work. Overlaying that with defects generated made the impact visible.

Understanding this impact allowed us, as a team, to put actions in place which will improve the way we approach these going forward.

 

We absolutely love running retros with our teams here at Plandek. They are that dedicated time you need to step away from your day-to-day work and focus on what we can learn about improving our team for the future.

Like all software teams, we’re constantly trying to improve and find software delivery metrics that help drive and measure our improvement. I’m yet to be convinced that there’s a level of “maturity” a team needs to reach before they can get value from metrics. Using metrics help us identify our challenges sooner and gives us something to measure against as we grow.

One of the things that we discussed this cycle was the impact that maintaining dependencies was having on progressing the work we’d committed to our customers.

In our last cycle, we weren’t able to deliver two important user stories that we’d committed to deliver. It’s always disappointing when you don’t deliver what was planned but equally, it’s really important to understand why? The why is never that teams aren’t working hard enough or putting the hours in. What was the reason we didn’t get to even start development on the stories we’d planned?

We looked into the data of what was completed and it was clear that the team had really been focussing on bugs being created by our QA team. Some of these bugs were a direct consequence of upgrading lots of pull requests being created by the ever-dependable Dependabot.

The chart above shows the number of dependabot-generated pull requests and the subsequent number of bugs created in the system. For January, we could see that there was:

  • A high number of dependencies to upgrade (who knew that January was the month to release updates?)
  • A high number of bugs caused by regressions from upgrading

As always the team had worked really hard, but they were being consumed by dependencies and associated bugs.

There was a further complication to this, as our Team Lead was ultimately being consumed by these dependencies as he’d long ago volunteered to be the one to manage and maintain all of them (something I’m sure is not uncommon among a lot of teams out there). However this meant we couldn’t deliver the 2 stories we’d committed.

The chart above shows the Dependabot pull requests created and a series for each of the engineers who either closed or merged the PR. Guess what colour our Team Lead is… 😂

Equipped with this new information we now had to think about what to do next? Upgrading and maintaining dependencies is a core part of good software development practice, but we want a little more control about when we do it and the impact it has on us delivering business value… so we have an initiative for next month, which is to :

  1. Share the (Pain) Love – Ensuring that more than one member of the dev team is managing the upgrades reduces the impact on one member of the team, a key member of our team too. It equally encourages professional growth across more of our team members.
  2. Deal with Dependencies More Frequently – Rather than allowing the backlog of dependency updates to grow, the team will triage and address them at shorter intervals (weekly rather than monthly). Smaller sets of changes reduce our risk of regressions.
  3. Prioritise Dependencies Against Features – When a dependency update has more complexity than normal (e.g. a major version upgrade, rather than a patch), we’re going to prioritise it against other features we’re trying to develop…
  4. Measure and Learn – We have both these metrics now and so in the coming weeks we’ll be looking at how we do on this initiative using the data from Plandek…

Finally, just a couple of points on the impact this initiative is hopefully going to have on some of our team :

Luke – Head of Product

This initiative will give us the ability to prioritise larger dependency updates along with everything else so there’s no surprise for the business

 

Edu – Lead Developer

This should ease the stress of having an additional workload that was hidden from the rest of the team while also being given more time to focus on the core work he wants to get done

 

Jef – Head of Engineering

The initiative gives us the opportunity to grow other members of the team into helping with dependency management, up-skilling them and sharing knowledge along the way. It will also increase visibility of necessary, but often unplanned, work impacting the throughput of the teams.

 

Maryna – QA Lead

I expect this to reduce the chance of having situations where the QA team have to fire-fight as we have updated everything at once.

share this post