Apply the defense test to AI conclusions
Before acting on AI-generated conclusions, apply the defense test: 'Could I defend this conclusion without the AI's output? Do I understand the reasoning well enough to identify where it might be wrong?'—if not, do the cognitive work before proceeding.
When This Fires
- Before acting on any AI-generated conclusion for a consequential decision
- When AI output "feels right" but you couldn't reconstruct the reasoning independently
- When using AI analysis for decisions that affect others, commit resources, or are irreversible
- Complements Scale AI output verification to stakes (stakes-based verification) with the epistemic independence test
Common Failure Mode
Treating AI as oracle: "The AI says we should do X → let's do X." If you can't explain why X is right, identify the assumptions X rests on, and articulate where X might fail, you're outsourcing judgment rather than augmenting it. AI should help you think better, not replace thinking.
The Protocol
(1) When AI generates a conclusion you're about to act on, pause. (2) Part 1 — Defense test: "Could I defend this conclusion to a skeptic without referencing the AI? Do I understand the reasoning chain well enough to explain it?" If no → the AI understood something you don't. Before acting, trace the reasoning yourself until you can defend it. (3) Part 2 — Vulnerability test: "Can I identify at least two specific ways this conclusion might be wrong — assumptions it relies on, data it might be missing, edge cases where it fails?" If no → you don't understand the conclusion's limits. Before acting, probe the weaknesses until you can articulate them. (4) If both tests pass → act. You understand and can evaluate the AI's contribution. (5) If either test fails → do the cognitive work before proceeding. The cost of understanding is always less than the cost of acting on a conclusion you can't evaluate.