Buy or Build: which is best for Software Engineering Intelligence Platforms?

To buy or build is a question often asked by tech leaders and engineering heads when looking at new software or digital tools – sometimes it makes sense to buy an existing platform (why reinvent the wheel?). There may be other times a more custom-built solution is needed to meet the requirements.

This question comes up time and time again for many of our larger clients when looking to invest in a Software Engineering Intelligence (SEI) platform like Plandek. Should they invest in a pre-built solution with ready-to-go metrics and dashboard templates, or build a bespoke analytics solution in-house which can grow with their future requirements? Perhaps there is a third option? *Yes, you can do both – keep reading to find out why you might choose this option!

Here’s our brief outline of the key considerations when assessing this question to help you avoid any pitfalls and choose the right solution for your organization, based on its Agile maturity, high-level goals and other key factors.

Buy or Build: the key success criteria for deciding to buy or build a Software Engineering Intelligence platform

We have listed our top six considerations when evaluating the buy or build decision for an software delivery analytics tool. All six considerations will need to be addressed in order to provide a solution for your teams that meets the two vital success criteria for adoption which are:

1. Teams TRUST the data and believe it meaningfully reflects the idiosyncrasies of their delivery process/workflows etc.
2. Teams, managers and leadership USE the tool as it provides new insight (not currently available in existing toolsets) that helps them: deliver milestones better today; and continuously improve their delivery effectiveness over time.

To do both of these things well, the devil really is in the detail!

Here’s the 6 key success criteria for choosing whether to buy or build a Software Engineering Intelligence (SEI) platform:

Consideration 1 – integration with complex system infrastructures, messy data, moving landscape – not suited to off-the shelf BI tools

Engineering Intelligence tools require the ability to analyse the end-to-end software delivery process which is underpinned by multiple, separate DevOps toolsets.  These legacy toolsets often involve multiple instances of tools both on-prem and in the cloud and often present a daunting target with which to integrate – especially as the usage of these tools change constantly over time. 

Moreover, these tools sets are often not used as they are designed to be (e.g. Jira) and the nature of the agile delivery process is that individual teams are often encouraged to adopt bespoke practices and workflows.   Hence the data is messy when viewed at the most disparate level and is potentially non-sensical when aggregated.

Consideration 2 – data enhancement and linkage of multiple data sources to identify insight

Much of the value of a tool like Plandek is its ability to follow a ticket (an increment of work) as it passes through the SDLC – from ideation to the live environment. This means linking a ticket reference in a workflow management system to an individual commit, in order to follow the same ticket – from backlog, to code commit, integration and deployment to the live environment.  This is a non-trivial task and is a pre-requisite to surface metrics such as Lead Time to Value.

Consideration 3 – customisable data visualisation and configurable metrics/analytics

A key challenge is to build an analytics tool that adds value at all levels of the delivery capability (team, manager and leadership).  As such, all metrics need to be:

• Highly configurable to accurately reflect the underlying processes/workflows – and the required measure
• Transparent so that method of calculation/configuration is visible to the user
• Statistically significant so outliers are removed (e.g. removing the 90th percentile of Average Age of Tickets)

Metrics then need to be configurable into customised dashboards for different personas at different levels in the organisation.  Plandek achieves this with its flexible Workspaces.

This ability to surface aggregated metrics presents a myriad of problems.  Aggregation requires the ability to aggregate messy data from multiple teams, workflows, sources – into meaningful aggregate metrics.  Plandek achieves this with unique capabilities such as ‘Status Groupings’.  

Organisations may have hundreds of teams each running with separate workflows and separate naming conventions for statuses in Jira.  Plandek enables these to be grouped into Status Groupings so that a Cycle Time (for example) can be calculated at an aggregate level.  Without this ability, surfacing meaningful metrics at an aggregate level becomes very tricky.

Consideration 4 – integrating other data sources from outside the SDLC

To provide powerful insight (particularly at the leadership level), users will want to understand how resource is allocated, what initiatives cost and how software delivery is impacting value delivery and customer satisfaction (for example).  This requires the ability to ingest data from many other data sources – most notably financial data, HR data and customer data.  

This is a core capability of Plandek and one which takes time and effort to replicate effectively.

Consideration 5 – adding prescriptive and intelligent data analytics to help teams deliver better

A key benefit of advanced Engineering Intelligence tools like Plandek, is the ability to move beyond descriptive data analytics (reviewing historical time series data of metrics), into the really interesting world of prescriptive data analytics, in order to answer such question as – when will this initiative be completed? And what can I do to deliver it more quickly without reducing scope or increasing resource?  

This requires a sophisticated data science capability, that takes into account the four considerations above, in order to answer questions like these usefully.

As such, Plandek has a function called SmartView. SmartView contains proprietary ML algorithms which can (for example) analyse a team’s complete ticket history and compare it to the current tickets being worked on within a sprint or epic, in order to:

• Identify anomalous tickets and potential blockers
• Quantify delivery risk (of a sprint, epic or initiative)
• Present actions to take to mitigate risk
• Forecast likely due date in the light of the identified risk and mitigations.

SmartView learns over time and becomes better at identifying mitigations (like a sort of super-powered Scrum Master!).  It is therefore very hard to replicate if you were to build a tool in-house.

Consideration 6 – the change management challenge.  What do we measure and how do we drive metrics-led improvement?

The final consideration is around delivering real business benefit from Engineering Intelligence tools.  This requires strong leadership sponsorship and a robust process of management support (normally provided by the Plandek Customer Success team).  Without the ongoing help of the Plandek domain experts, an organisation would have to:

• Correctly select the first and second order metrics that best reflect your objectives, level of Agile maturity and delivery process
• Build your analytics tool accordingly
• Roll-out the tool and coach teams on its use
• Develop and share benchmarks

All of our client research shows that a key motivator for choosing to partner with Plandek is this vital level of support provided – to help engage teams and drive the change desired as rapidly as possible.  

Buy or Build: how to choose the right option for your software engineering intelligence platform

Deciding whether to buy to build your software delivery analytics engine is a big decision. Luckily, Plandek makes it possible to reap the rewards of both options.

Plandek allows you to build your own custom solution inside our platform (much like a powerful BI tool), while giving you to benefits of out-the-box metrics and dashboards too.

• Save time, with out-the-box metrics and dashboards already built for you. Get useful insights within just minutes of getting started, not weeks.
• Get inspired and learn from existing templates based on popular frameworks (like DORA).
• Change or expand upon existing metrics or dashboards – Plandek is flexible, with literally hundreds of metrics configuration combinations to play with.
• Start from scratch, if you prefer a blank page. Build exact what you need from a dashboard.

Plandek is the middle-ground between building your own platform from scratch and using a traditional SEI platform with more limited features and functionality. Why not talk to a member of our team to see just how customizable Plandek is and what you can do with the tool? Book a demo to get the conversation started.