Give each workflow change 3-5 executions before deciding to keep, modify, or revert — distinguish signal from noise and prevent oscillation

Why This Is a Rule

Workflow performance varies from execution to execution due to common-cause variation (Collect at least 5 measurements before improving a workflow — distinguish common-cause variation (inherent) from special-cause variation (fixable)). A change that appears to help after one execution might simply be normal fluctuation — the next execution would have been better anyway. Similarly, a change that appears to hurt after one execution might be a coincidental bad day. Reacting to single data points produces oscillation: change → seems bad → revert → seems bad → change again → seems bad → revert again. The workflow thrashes between states without ever settling into either.

Three to five executions is the minimum sample that separates persistent effects from noise. If the change consistently helps across 3-5 executions (metrics improve or hold in each), it's likely a genuine improvement. If it consistently hurts, it's genuinely harmful. If results are mixed (helps sometimes, hurts sometimes), the change is probably neutral and you can keep or drop it without much consequence.

This patience threshold pairs with Exactly one improvement per execution cycle — not zero, not three — so you can attribute changes to effects's one-change-per-cycle rule: you make one change, then observe it for 3-5 cycles before concluding anything. This means you're running a controlled experiment with adequate sample size rather than reacting to noise. The total cost is 3-5 executions per experiment; the benefit is reliable attribution and no oscillation.

When This Fires

• After implementing a workflow change and evaluating whether it worked
• When tempted to revert a change after a single bad execution
• When workflow improvements seem to "not stick" — possibly reacting to noise
• Complements Exactly one improvement per execution cycle — not zero, not three — so you can attribute changes to effects (one improvement per cycle) with the evaluation patience threshold

Common Failure Mode

Impatient reversion: "I tried the new template and the first report took longer, so I reverted." The first execution with any change is often slower due to unfamiliarity. The second and third executions, with the learning curve past, might be significantly faster. Reverting after one data point discards changes that need a short learning period to show their value.

The Protocol

(1) After implementing a change (Exactly one improvement per execution cycle — not zero, not three — so you can attribute changes to effects), commit to running the workflow 3-5 times before evaluating. (2) Record the key metrics (Track workflow quality with tension-paired metrics (cycle time + error rate + energy cost) — single-metric optimization corrupts the system) for each execution. (3) After 3-5 executions, review the pattern: Consistent improvement (3+ of 5 executions show better metrics) → keep the change. Consistent degradation (3+ of 5 executions show worse metrics) → revert. Mixed results (roughly equal better/worse) → the change is likely neutral; keep it if it feels better, revert if it doesn't. (4) Do not make additional changes during the evaluation period — that would confound your experiment (Exactly one improvement per execution cycle — not zero, not three — so you can attribute changes to effects). (5) After concluding the evaluation, identify the next single improvement and restart the cycle.