Before finalizing a schema update, verify the new version…

When This Fires

• Before finalizing any schema revision triggered by disconfirming evidence
• After generating a new version of a belief or operating rule
• During schema evolution when the new version feels better but hasn't been regression-tested
• Any time you're replacing one mental model with another

Common Failure Mode

Testing the new schema only against the case that broke the old one: "The new schema explains the failure, so it's better." But does it also explain the five successes? A schema that explains one failure while failing to explain five successes is a net loss. The new schema must be a superset — handling everything the old one handled plus the new case.

The Protocol

Before finalizing a schema update: (1) List five specific situations where the old schema produced accurate, useful results. Choose diverse cases across the schema's range. (2) For each, verify: does the new schema also handle this case correctly? Does it produce the same (accurate) prediction or recommendation? (3) If all five pass → the new schema is backwards-compatible. It handles the new case AND preserves old successes. Finalize the update. (4) If any case fails → the update introduces a regression. Refine the new schema until it handles both the new case and all five old cases.