Measuring Engineering Productivity Without Vanity Metrics

The history of engineering productivity measurement is a graveyard of well-intentioned metrics that incentivized the wrong behavior. Lines of code rewarded verbosity. Story points rewarded point inflation. Commit counts rewarded micro-commits. Each metric was easy to measure and completely uncorrelated with actual delivery capability.

The DORA metrics are a starting point, not the answer

The four DORA metrics, deployment frequency, lead time for changes, mean time to recovery, and change failure rate, are a significant improvement over vanity metrics. They measure outcomes rather than activities. But they are trailing indicators that tell you how you performed, not leading indicators that predict how you will perform. And they say nothing about the quality of the architecture, the sustainability of the codebase, or the long-term maintainability of the system.

Delivery capability metrics

The metrics that actually predict long-term delivery capability are structural: code modularity, dependency health, test value (not just coverage), architecture decision traceability, and knowledge distribution. These metrics tell you whether the system is getting easier or harder to change over time.

• Code modularity: what percentage of changes touch more than one service boundary
• Dependency health: how many dependencies are outdated, vulnerable, or unmaintained
• Test value: how many production bugs were caught by the test suite before deployment
• Decision traceability: what percentage of architecture patterns have documented rationale
• Knowledge distribution: how many engineers can confidently change each part of the system