Metrics
11. Intelligent analytics for software delivery
Much of the discussion in delivery and engineering metrics concerns ‘descriptive’ data analytics – i.e. the presentation of metrics, such as the DORA or Flow metrics, that can be tracked over time to provide insight.
These descriptive analytics are a big step forward for many organisations as they can help teams drive very significant improvement in Agile software delivery performance.
However, analysis of these metrics is not ‘intelligent’ in that it requires the user to assess the metrics in order to make their own predictions, assess risk and identify causes and mitigations.
There is a very big opportunity, therefore, to provide truly intelligent analytics to software delivery teams.
Indeed, in the context of optimising the SDLC, the application of machine learning and/or AI is enabling the provision of:
Predictive analytics – to start to predict the delivery of sprints, epics, etc.
Prescriptive analytics – to identify actions and mitigations to correct course and improve delivery performance/outcomes.
Team-level intelligent analytics to improve sprint and epic delivery
Applying intelligent analytics to the Agile delivery process is constrained by the quality of the underlying data available (e.g. the quality of data residing in workflow management tools like Jira).
It is also constrained by the nature of Agile, whereby much of a ‘project’ or product will remain undefined and un-estimated, limiting the available data set from which to derive intelligent analytics (e.g. predicting a software delivery date).
Despite these constraints, intelligent analytics can provide huge benefits to software delivery teams, particularly at the team/sprint level.
The example below (Figure 31) shows a view from Plandek’s SmartView intelligent analytics tool at the sprint level.
The SmartView platform analyses the entire ticket history of the team in question to compare it to the current ticket set in order to identify anomalies. Having identified meaningful anomalies, SmartView’s algorithms quantify risk and identify ‘Action items’ (mitigations) that can be taken to correct course and improve sprint accuracy.
‘Action items’ point teams to the tickets that are outside the expected parameters. The parameters considered include:
- Ticket stalling: when tickets appear to stall at any point throughout the delivery process or exceed standard cycle times and require immediate action by the team.
- Development complexity: commit size/count, files changed, and other parameters exceed the norm for tickets of similar size and complexity.
- Pull request anomalies: when pull requests stall or indicate a higher-than-normal level of activity, suggesting a team is struggling to merge.
- Build event anomalies: when the builds containing the related tickets fail or succeed.
- Deployment anomalies: when tickets are deployed or if the deployment fails and requires action.
SmartView is an example of intelligent analytics supplementing the expertise of the Team Lead/Scrum Master by reviewing a very large amount of data and identifying anomalies or blockers that are not necessarily visible ‘to the naked eye’.
It is also providing:
- A quantitative risk assessment for sprint completion can be aggregated to consider risk at the broader epic or release train level.
- Suggested ‘action items’ or mitigations to improve sprint outcomes (prescriptive data analytics). As the algorithms improve, so too does the quality and breadth of these system-generated action items.
A similar principle can be applied to epics (see Figure 32 below).
Again, the algorithms look for issues that create risk and require mitigation. These data can be used to predict epic timing and to reduce potential delays.
The quant risk data can also be aggregated across products or release trains (for example) to better understand delivery risk across roadmap items.
