Retire metrics that can't distinguish gaming from genuine…

When This Fires

• When Audit decision-driving metrics every 3 months's gaming check reveals that gaming behavior and genuine progress produce the same metric reading
• When a metric consistently shows "good" results but the underlying reality feels wrong
• When you can describe exactly how to game a metric and your behavior matches that description
• During metric audits when determining whether to revise or retire

Common Failure Mode

Revising rather than retiring: "The metric has problems, so let's adjust the formula." If the fundamental proxy relationship has broken — if the metric can't distinguish real from gamed — no formula adjustment fixes it. You need a new metric that measures a different aspect of the outcome, one that gaming behavior hasn't yet learned to optimize. Revising a decoupled metric just makes the decoupled metric more complex.

The Protocol

(1) During metric audit (Audit decision-driving metrics every 3 months), apply the distinguishability test: describe what gaming this metric looks like. Describe what genuine progress looks like. (2) If you can't distinguish the two descriptions — if gaming behavior and genuine progress produce the same metric values — the metric has failed the distinguishability test. (3) Retire the metric immediately. Remove it from dashboards, stop tracking it, stop referencing it in decisions. (4) Accept the temporary information vacuum. "We don't know how to measure this right now" is an honest state that motivates finding a better metric. (5) Design a replacement metric that measures a different aspect of the outcome — one that gaming hasn't adapted to yet. The replacement will also eventually decouple (Goodhart's Law guarantees it), but it buys another 3-6 months of valid signal before the next audit.