After 15-20 judgments, check if your errors are bias…

When This Fires

• You've made enough project estimates to check your track record
• You've accumulated hiring decisions, performance predictions, or risk assessments in one domain
• You notice your judgments seem "off" but can't tell if you're biased or just inconsistent
• Any domain where you make repeated judgments of the same type

Common Failure Mode

Treating all errors as bias. You overestimated three projects in a row and conclude "I'm always too optimistic." But three data points isn't enough — and the next three might be underestimates. Without enough data to distinguish bias from noise, you apply a correction factor that makes half your estimates worse while improving the other half.

The Protocol

After 15-20 judgments in the same domain: (1) List each judgment alongside the actual outcome. (2) Calculate the error (judgment minus actual) for each. (3) Check directionality: do errors cluster positive (overestimate) or negative (underestimate), or do they scatter both ways? (4) If directional → calculate the average error and apply it as a correction factor to future judgments. If scattered → use aggregation strategies: multiple independent estimates, structured frameworks, or checklists that reduce noise.