Validate new feedback mechanisms in the first cycle
For each feedback mechanism you build, verify within the first cycle that the data reveals something unknown, that measurement effort is sustainable, and that you can specify one concrete adjustment based on results—if any component fails, redesign before continuing.
When This Fires
- After the first cycle of any newly built feedback mechanism
- When deciding whether to continue investing in a tracking system after initial deployment
- During feedback mechanism design reviews — build the three checks into the validation plan
- When a tracking system "doesn't feel worth it" after initial use — formalize the evaluation with these three checks
Common Failure Mode
Continuing a feedback mechanism past the first cycle despite failing one or more checks: "I'll give it more time — maybe it'll get better." If the mechanism doesn't reveal unknowns, isn't sustainable, or doesn't produce actionable adjustments in the first cycle, more cycles won't fix the fundamental design. Redesign immediately rather than investing effort in a flawed mechanism.
The Protocol
(1) After the first complete cycle of a new feedback mechanism, run three checks: Novelty: did the data reveal something I didn't already know? If no → the mechanism isn't adding information. Redesign to measure something that varies in ways you can't predict. Sustainability: was the measurement effort manageable? Could I sustain this for 3 months without heroic effort? If no → simplify the measurement until it's effortless (Accountability reporting must be near-zero effort…). Actionability: based on this data, can I specify one concrete behavioral adjustment? If no → the data isn't connected to adjustable behavior. Redesign to measure something you can actually change. (2) If all three pass → continue. The mechanism is viable. (3) If any fails → redesign the specific failing component before the second cycle. Don't proceed with a known-flawed mechanism.