Validate patterns with three filters
Before concluding a pattern is meaningful, verify it survives three independent filters: sample size check (occurrences vs. opportunities), base rate comparison (frequency vs. background rate), and alternative explanation generation (minimum two alternatives).
When This Fires
- After identifying a pattern and before acting on it
- When a perceived pattern is driving a decision (schedule change, investment, behavior modification)
- During any self-analysis where "I always..." or "this never..." claims emerge
- Before recommending a personal pattern to others
Common Failure Mode
Running one filter and skipping the others. You check sample size (enough occurrences) but skip base rate (the pattern occurs at background frequency) and alternatives (your preferred explanation isn't tested). Each filter catches a different type of error. All three must pass.
The Protocol
Before concluding a pattern is meaningful: (1) Sample size: list occurrences AND opportunities. Calculate the actual rate. Is it high enough to be non-random? (2) Base rate: what is the background frequency of this phenomenon without the suspected cause? Is your pattern significantly above baseline? (3) Alternative explanations: generate at least two alternative explanations for the same data. Can you rule them out? (4) Only patterns surviving all three filters → act on. Patterns failing any filter → gather more data or revise interpretation.