What is calibration feedback?

Calibration feedback is a concept in personal epistemology: You cannot improve the alignment between your confidence and your accuracy without external data that reveals the gap between what you believed and what actually happened. Calibration without feedback is guesswork about guesswork.

Example: A product manager estimates a 90% probability that a new feature will increase user retention. She ships it. Retention drops. Without tracking this prediction against the outcome — without feedback — she would never update her mental model. She would continue shipping features with the same misplaced certainty, accumulating a pattern of confident failure invisible to her own introspection. But she keeps a prediction log. After six months and forty tracked predictions, she discovers that events she rated at 90% confidence actually occurred only 60% of the time. The gap is enormous and would have been invisible without the log. She recalibrates. Over the next six months, her 90% predictions start landing at 82%. Still imperfect. But the feedback loop is compressing the gap — not through willpower or intelligence, but through the simple act of comparing what she believed to what happened.

This concept is part of Phase 8 (Perceptual Calibration) in the How to Think curriculum, which builds the epistemic infrastructure for perceptual calibration.