Three verification layers
Design verification as three independent layers—continuous signals (daily metrics), periodic samples (weekly/monthly spot-checks), and infrequent structural audits (quarterly full reviews)—with each layer optimized for different failure detection at different resource costs.
When This Fires
- When designing verification or quality assurance for any important system
- When a single verification method keeps missing certain failure types
- When building delegation oversight (Classify every task as ONLY ME, COULD DELEGATE, or SHOULD…-566) and needing multi-level quality assurance
- When a system fails despite "having monitoring" — the monitoring was probably single-layer
Common Failure Mode
Single-layer verification: relying only on daily metrics. The metrics show green while slow drift (detectable only by periodic sampling) or structural degradation (detectable only by audit) accumulates unseen. By the time the drift becomes acute enough for daily metrics to detect, significant damage has occurred.
The Protocol
(1) For each important system, design three independent verification layers: Continuous (daily): what automated metrics track this system's health? Define 2-3 key metrics with alert thresholds. Cost: near-zero (automated). Periodic (weekly/monthly): what should you manually sample or spot-check? Define the sampling protocol — what to check, how many samples, what quality criteria. Cost: 15-30 minutes per period. Structural (quarterly): what comprehensive review validates the system's architecture and assumptions? Define the audit scope and questions. Cost: 1-3 hours per quarter. (2) Each layer should be independent: if one layer fails to function, the others still operate. (3) Each layer catches failures the others miss: continuous catches acute, periodic catches drift, structural catches architectural. (4) Trust the layers: if all three show green, confidence is high. If any one shows problems, investigate regardless of what the other layers show.