Get started

Blog

Metric of the week: Commits Without a Pull Request

Plandek, April 12, 2021

Plandek Metric of the Week: Commits Without a Pull Request

Commits Without a Pull Request (CWPR) is an important engineering metric for all organisations and particularly those focused on software engineering quality and DevSecOps.

It measures all merges to master that have not been peer reviewed (i.e. do not have a related pull request). This is generally considered as poor engineering practice and a potential security concern, particularly in organisations with engineers of variable experience and varied familiarity with the code base.

CWPR rates may vary widely between teams, locations, contractors etc – and hence it is a key software engineering quality metric. As such, software delivery teams at a higher level of Agile DevOps maturity will routinely focus on CWPR is a part of their broader engineering quality management process.

It is also a popular metric for delivery teams focused on DevSecOps, as effective DevSecOps recognises that teams and engineers themselves need to consistently put engineering security at the heart of their daily routines and processes.

Example Commits without a Pull Request summary chart – Plandek Delivery and Engineering Quality Dashboard

Example Commits without a Pull Request breakdown chart – Plandek delivery and engineering quality dashboard

As shown in the expanded view of CWPR above, the measure allows Teams Leads and Engineering Managers to review CWPR rates by committer and repo over time. The advanced filter functionality within Plandek also enables more detailed breakdowns such as by ticket issue type, ticket labels, and ticket status.

The expanded view therefore enables teams to rapidly identify where the problems lie and to improve CWPR levels. For some organisations, 100% compliance (i.e. a CWPR level of 100%) would be the minimum target. In other organisations, CWPR targets may be set by project, team, location etc. Our experience shows that typically a CWPR rate of over 10% is seen as a serious concern.

Related metrics

Commits Without a Pull Request is often used in conjunction with other delivery and engineering quality metrics. Commits Without a Ticket Reference is another related engineering quality metric. It tracks the percentage of code commits made to any branch that do not have a related (e.g. Jira) ticket reference. The ticket reference is required for effective root-cause analytics – to trace the links between tickets and code commits.

Other delivery and engineering quality metrics include measures of software quality such as: Escaped Defects, Bug Resolution Time and Unresolved Bugs.

Key use cases

Commits Without a Pull Request is used to monitor a key discipline the underpins engineering process quality – namely the peer review of code before it is committed by an individual engineer.
As such, it is a key engineering quality metric that needs careful monitoring to ensure both code quality and code security, as part of good DevSecOps practice. More broadly it is a good indicator metric of general process adherence and security disciplines within a software delivery team.

Tracking Commits Without a Pull Request may be particularly useful for:

  1. Immature delivery teams new to adopting Agile DevOps methodologies and tools
  2. Distributed software delivery teams consisting of a wide variety of engineer types: in-house, contractor, onshore, offshore etc. As such environments may involve a higher turnover of engineers and many who are unfamiliar with the code base
  3. Complex, strategically critical software delivery projects involving the build of new applications and features
  4. Businesses with robust regulatory requirements and/or very high risk profiles.

Expected outcomes

By reducing Commits Without a Pull Request, delivery teams will significantly reduce a key potential source of quality and infosec risk. In certain circumstances, regulatory requirements will mandate that CWPRs are tracked and reported on, in order to demonstrate effective quality and infosec procedures.

share this post