Search logs for evidence that falsifies your hypothesis, not confirms it
When debugging with strong initial hypotheses about root cause, deliberately search logs for evidence that would falsify the hypothesis rather than confirm it, to counteract confirmation bias in data collection.
Why This Is a Rule
When you believe you know the root cause, every log entry gets filtered through that hypothesis. Entries consistent with the hypothesis feel like confirmation. Entries inconsistent with it get explained away: "that's probably unrelated" or "the timing is coincidental." This confirmation bias in data collection means the hypothesis can survive even when the evidence doesn't support it — because you're only looking at evidence that does.
Deliberate disconfirmation search reverses the bias: instead of asking "does this log entry support my hypothesis?", ask "what log entry would I expect to see if my hypothesis is wrong?" Then specifically search for that entry. If you find it, the hypothesis needs revision. If you don't find it after genuine search, the hypothesis is stronger — not because you confirmed it, but because you tried to falsify it and failed.
This is Popper's falsificationism applied to debugging: hypotheses gain credibility not through accumulating confirmations but through surviving genuine attempts at falsification.
When This Fires
- During debugging when you form a strong initial theory about root cause
- When the first log entry you see "confirms" your hypothesis (the confirmation is probably selection bias)
- In incident response when the team quickly converges on a theory
- Any investigation where the hypothesis formed before comprehensive data collection
Common Failure Mode
Searching for confirming evidence with the label "I'm just verifying." Verification and falsification feel similar but produce different search behaviors. Verification: "Let me find the log entry showing the database timeout." Falsification: "If my hypothesis is wrong, I'd expect to see [specific alternative pattern] — let me search for that." The first search confirms; the second tests.
The Protocol
When you have a strong debugging hypothesis: (1) State the hypothesis explicitly. (2) Ask: "If this hypothesis is wrong, what evidence would I expect to see?" (3) Search specifically for that evidence — the disconfirming pattern, not the confirming one. (4) If you find disconfirming evidence → revise the hypothesis. (5) If you genuinely search and don't find it → the hypothesis is strengthened through survived falsification, not through selected confirmation.