2. “Metrics Lemmings”

As well as Method Lemmings, I would like to propose Metrics Lemmings, where so-called “Agility Metrics” (such as, rate of releases, number of commits) are imposed top-down to get staff to behave in a “more agile way”, despite the fact that these can be fairly easily gamed.

In Accelerate, Nicole Fosgren attempts to measure the success of IT projects, and correlate those with individual practices. Fosgren measures:

  • Lead Time (time to get a new change delivered)
  • Deployment Frequency
  • Mean Time To Restore (how quickly a team recover from an outage)
  • and Change Fail Percentage.

She then goes on to correlate these measures with the financial success of the organisations being studied. Now, it’s unlikely that Fosgren believes

We measured product delivery lead time as the time it takes to go from code committed to code successfully running in production, and asked survey respondents to choose from one of the following options: We applied cluster analysis in all four years of the research project and found that every year, there were significantly different categories of software delivery performance in the industry.

Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.

Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.

“We conducted additional research and found that teams using branches that live a short amount of time (integration times less than a day) combined with short merging and integration periods (less than a day) do better in terms of software delivery performance than teams using longer-lived branches.” - Nicole Forsgren, Accelerate.

There are of course, problems with this. But, it’s a short jump from this kind of well-meaning research to these metrics being used to measure performance of developers, leading to Goodhart’s Law:

“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” - Goodhart’s Law, Wikipedia

Add Your Star On GitHub to receive an invite to the GitHub Risk-First team.

Rob Moffat
Rob Moffat Author of Risk-First Software Development. Developer. Working in the UK.