Plandek Metric of the Week: Commits Without a Pull Request
Commits Without a Pull Request (CWPR) is an important engineering metric for all organisations and particularly those focused on software engineering quality and DevSecOps.
It measures all merges to master that has not been peer-reviewed (i.e. do not have a related pull request). This is generally considered as poor engineering practice and a potential security concern, particularly in organisations with engineers of variable experience and varied familiarity with the code base.
CWPR rates may vary widely between teams, locations, contractors etc – and hence it is a key software engineering quality metric. As such, software delivery teams at a higher level of Agile DevOps maturity will routinely focus on CWPR as a part of their broader engineering quality management process.
It is also a popular metric for delivery teams focused on DevSecOps, as effective DevSecOps recognises that teams and engineers themselves need to consistently put engineering security at the heart of their daily routines and processes.
As shown in the expanded view of CWPR above, the measure allows Teams Leads and Engineering Managers to review CWPR rates by committer and repo over time. The advanced filter functionality within Plandek also enables more detailed breakdowns such as by ticket issue type, ticket labels, and ticket status.
The expanded view, therefore, enables teams to rapidly identify where the problems lie and to improve CWPR levels. For some organisations, 100% compliance (i.e. a CWPR level of 100%) would be the minimum target. In other organisations, CWPR targets may be set by project, team, location etc. Our experience shows that typically a CWPR rate of over 10% is seen as a serious concern.
Related metrics
Commits Without a Pull Request is often used in conjunction with other delivery and engineering quality metrics. Commits Without a Ticket Reference is another related engineering quality metric. It tracks the percentage of code commits made to any branch that does not have a related (e.g. Jira) ticket reference. The ticket reference is required for effective root-cause analytics – to trace the links between tickets and code commits.
Other delivery and engineering quality metrics include measures of software quality such as Escaped Defects, Bug Resolution Time and Unresolved Bugs.
Key use cases
Commits Without a Pull Request is used to monitor a key discipline that underpins engineering process quality – namely the peer review of code before it is committed by an individual engineer.
As such, it is a key engineering quality metric that needs careful monitoring to ensure both code quality and code security, as part of good DevSecOps practice. More broadly it is a good indicator metric of general process adherence and security disciplines within a software delivery team.
Tracking Commits Without a Pull Request may be particularly useful for:
- Immature delivery teams who are new to adopting Agile DevOps methodologies and tools.
- Distributed software delivery teams consisting of a wide variety of engineer types: in-house, contractor, onshore, offshore etc. As such environments may involve a higher turnover of engineers and many who are unfamiliar with the code base.
- Complex, strategically critical software delivery projects involving the building of new applications and features.
- Businesses with robust regulatory requirements and/or very high-risk profiles.
Expected outcomes
By reducing Commits Without a Pull Request, delivery teams will significantly reduce a key potential source of quality and infosec risk. In certain circumstances, regulatory requirements will mandate that CWPRs are tracked and reported on, in order to demonstrate effective quality and infosec procedures.
