Skip to content
How to ThinkIn the Age of AI

Define rollback metrics and automated reversal triggers…

When implementing feature flags, canary deployments, or A/B tests, treat the deployment decision as a two-way door by defining rollback metrics and automated reversal triggers before deployment.

When This Fires

  • Before any deployment that uses feature flags, canary processes, or A/B testing
  • When designing deployment pipelines and wanting to embed reversibility
  • When a deployment goes sideways and no one can decide whether to roll back — this rule should have fired earlier
  • When applying Before committing to a one-way door, search for ways… (restructure one-way doors) to software engineering decisions

Common Failure Mode

Deploying with feature flags but no rollback criteria: "We can always roll back if it goes badly." Without defined "badly," you'll debate whether the current state is bad enough while the incident escalates. The feature flag is technically a two-way door, but operationally it's a one-way door because the reversal decision requires real-time deliberation under crisis conditions.

The Protocol

(1) Before deployment, define rollback metrics: which metrics, what thresholds, over what time window. Be specific: "error rate >2% sustained for >5 minutes" not "if errors increase significantly." (2) Where possible, automate the reversal: set up monitoring that triggers automatic rollback when thresholds are breached. (3) Define the observation window: how long after deployment before evaluating? Minutes for canary, hours for phased rollout, days for A/B tests. (4) During the observation window, watch defined metrics only. Don't get distracted by metrics that weren't in the pre-defined set. (5) If rollback triggers fire → roll back immediately. Do not override automated rollback without a specific, documented reason. The whole point of pre-defining criteria was to prevent crisis-state judgment from overriding rational analysis.