Monitor your error detection system itself — track what…
Build error detection infrastructure that monitors both your primary outputs and your detection system's own performance, tracking what errors you catch versus what you miss through other means to detect detection failures.
When This Fires
- When building any error detection or quality assurance system
- When you discover an error through an unexpected channel — log it as a detection miss
- During detection system audits when evaluating coverage
- When you have high confidence in a detection system but haven't verified it empirically
Common Failure Mode
Assuming detection works because you haven't seen errors: "Our review process must be catching everything because nothing is slipping through." This confuses absence of evidence with evidence of absence. If your detection system misses errors and no other channel catches them either, you'll never see the errors — they silently degrade quality while you maintain false confidence.
The Protocol
(1) For each detection system (code review, financial review, behavioral agent audit, health monitoring), track two categories: Caught: errors detected by the system as designed. Escaped: errors discovered through other channels (customer reports, colleague observations, random discovery, downstream failures). (2) Calculate detection coverage: caught ÷ (caught + escaped). (3) If coverage < 80% → the detection system has significant blind spots. Analyze the escaped errors: what type were they? What channel revealed them? Why didn't the primary system catch them? (4) Improve the detection system to cover the escaped error types. (5) Re-measure after improvement. Detection coverage should increase with each cycle of meta-detection.