Assign criterion weights before scoring options — knowing scores first lets you unconsciously rig the weights
When building a decision matrix, assign criterion weights before scoring any options to prevent unconscious adjustment of weights toward your pre-existing preference.
Why This Is a Rule
If you score options first and then assign weights, a predictable bias emerges: you unconsciously increase the weight of criteria where your preferred option scores well and decrease the weight where it scores poorly. "Well, location isn't really THAT important" — said after discovering your preferred apartment scored poorly on location. This isn't deliberate manipulation; it's confirmation bias operating through the weight-assignment mechanism. The result is a matrix that mathematically confirms the choice you'd already made intuitively, dressed in the costume of rigor.
The temporal firewall — weights before scores — prevents this by making weight assignment blind to the scoring outcomes. When you assign weights, you don't yet know which option will benefit from each weight. You're forced to evaluate criteria importance based on genuine priority rather than backward-fitting from a preferred outcome.
This is the same principle behind blind reviewing in science and blind auditions in orchestras: removing information that biases evaluation. In this case, the information being blinded is which option each weight will favor.
When This Fires
- Every time you build a decision matrix (Use a weighted decision matrix when options exceed 3 and criteria exceed 4 — working memory cannot hold all dimensions at once) — this is a mandatory sequencing step
- When you notice yourself wanting to revisit weights after seeing scores — that's the bias trying to operate
- When stakeholders disagree on weights during group decisions — have them assign independently before revealing scores
- During any structured evaluation where criteria weights interact with scoring
Common Failure Mode
"Let me just see the scores first to get a sense of things, then I'll finalize the weights." This exactly inverts the required sequence. Once you've seen scores, you can't unsee them. Every weight assignment is now contaminated by knowledge of which option each weight helps. The matrix becomes a rationalization tool rather than a decision tool.
The Protocol
(1) Before scoring any option on any criterion, assign weights to all criteria. Ask: "How important is this criterion relative to the others, independent of any specific option?" (2) Lock the weights. Write them down. Do not adjust them after scoring begins. (3) If you genuinely believe a weight was wrong after seeing scores → you must articulate the reason that's independent of any specific option's performance. "Location matters more because I'll commute daily" is legitimate. "Location matters less because my favorite option is far away" is bias. (4) For group decisions: have each person assign weights independently before any scoring occurs. Aggregate weights, then score collectively. (5) Treat the urge to adjust weights after scoring as diagnostic: it likely means your preferred option didn't win analytically, and the matrix is doing its job of challenging your pre-existing preference.