Skip to content
How to ThinkIn the Age of AI

Classify trigger firings as true/false positives for one…

Log each trigger firing for one week as true positive or false positive, then adjust the threshold only after accumulating empirical data rather than based on single instances.

When This Fires

  • After deploying any new trigger, before making the first calibration adjustment
  • When tempted to adjust a trigger because "it fired at the wrong time today"
  • During the initial calibration phase of any behavioral agent (first 2-4 weeks)
  • When a trigger feels "off" but you can't articulate whether it's too sensitive or not sensitive enough

Common Failure Mode

Adjusting after every frustrating misfire: "The trigger went off during my meeting — I'll change the conditions." Maybe meetings are a genuine false-positive pattern (change is warranted). Maybe it was a one-time scheduling overlap (change would overfit). You can't distinguish these without a week of data showing whether meeting-time firings are rare or systematic.

The Protocol

(1) For one week, log every trigger firing with a binary classification: TP (true positive — the trigger fired and you should have acted) or FP (false positive — the trigger fired but the action wasn't appropriate for this moment). (2) Also log suspected FN (false negatives — moments the trigger should have fired but didn't). (3) After one week, compute: TP rate = TP / (TP + FP). This is your trigger's precision. (4) If precision ≥ 80% → trigger is well-calibrated. Leave it alone. (5) If precision < 80% → identify the pattern in false positives. What context produced them? Add a guard clause for that context (When false positives exceed 30%, add guard clauses…). (6) If false negatives are common → the trigger is too conservative. Gradually increase sensitivity (Start triggers conservatively — 3-5 daily activations… in reverse).