A Decision Matrix That Does Not Pretend to Be Objective

DECISION MATRIX IN 6 STEPS 1. List your options (rows). 2. List the criteria that actually separate the options (columns). If every option scores the same on a criterion, drop it — it adds weight but no information. 3. Filter dealbreakers FIRST. A must-have/must-not-have is a gate, not a weight. "Must be audit-compliant" = disqualify, don't discount. 4. Assign weights (importance) that sum to 100. This is an opinion — say so out loud. 5. Score each option per criterion on 1-5, with a one-line justification for each score. A score you can't defend is noise. 6. Weighted total = sum(weight x score). Then do a sensitivity pass (below) before trusting the ranking. WORKED EXAMPLE: Waterfall vs Scrum vs Kanban Criteria: Adaptability to change, Schedule/cost predictability, Low process overhead, Ease of adoption, Stakeholder transparency. Raw scores (1-5), same in both scenarios: Adapt Predict Overhead Adoption Transparency Waterfall 1 5 2 3 3 Scrum 5 3 2 3 4 Kanban 4 3 5 4 3 Scenario A — startup, uncertain requirements. Weights: Adapt 35, Predict 15, Overhead 20, Adoption 15, Transparency 15. Waterfall: 1x.35 + 5x.15 + 2x.20 + 3x.15 + 3x.15 = 2.40 Scrum: 5x.35 + 3x.15 + 2x.20 + 3x.15 + 4x.15 = 3.65 Kanban: 4x.35 + 3x.15 + 5x.20 + 4x.15 + 3x.15 = 3.90 <- wins Scenario B — regulated enterprise, fixed-price contracts. Weights: Adapt 10, Predict 40, Overhead 10, Adoption 15, Transparency 25. Waterfall: 1x.10 + 5x.40 + 2x.10 + 3x.15 + 3x.25 = 3.50 <- wins Scrum: 5x.10 + 3x.40 + 2x.10 + 3x.15 + 4x.25 = 3.35 Kanban: 4x.10 + 3x.40 + 5x.10 + 4x.15 + 3x.25 = 3.45 HOW WEIGHTS CHANGE THE RESULT Same scores, different weights, completely different winner: Kanban in A, Waterfall in B. Nothing about the options changed — only your priorities did. That's the whole point and the whole danger. The matrix doesn't find the truth; it computes the consequence of your assumptions. Note also: in Scenario B the top two are 3.50 vs 3.45 — a 0.05 gap. That is not a win, it's a tie. When the margin is smaller than your scoring resolution (each score is a 1-point integer, so ±0.3 of noise is baked in), the ranking is meaningless and you need a real tiebreaker: cost, team preference, pilot test. WHEN NOT TO TRUST THE SCORE 1. When scores are guesses. A 3 with no evidence is an opinion wearing a number. If you can't write the justification, the score is fake precision. 2. When criteria overlap. "Overhead" and "ease of adoption" partly measure the same thing — double counting inflates whichever option is good at both. 3. When weights were chosen after the favorite was picked. That's not analysis, it's rationalization. Set weights before you look at scores, or have someone who doesn't know the options assign them. 4. When the gap is small. Anything within ~0.3 of the leader is a tie. Declare it a tie and stop pretending. 5. When a dealbreaker is buried as a weight. A 20% weight on "compliance" lets a non-compliant option still score 4.0 overall. It should have been eliminated in step 3. 6. When you're mixing apples and dollars. Cost in dollars, morale on 1-5, risk as a percentage — all forced onto one 1-5 scale. That conversion is the most subjective step in the whole exercise and the easiest to hide. 7. When the decision is cheap and reversible. If you can change course in a week, don't spend a day weighting criteria. The matrix's precision is fake where stakes are low. 8. When the score contradicts a visible trade-off. If Waterfall "wins" but you can see it scored 1/5 on adaptability and your requirements are volatile, the matrix is telling you your weights are wrong, not that Waterfall is right. The score is a prompt to argue, not a verdict. HOW TO KEEP THE SUBJECTIVITY VISIBLE (not hidden) - Show raw scores AND weighted totals side by side, so the trade-off is visible, not buried in one number. - Record who set each weight and why. If two people disagree on a weight, discuss it — never average the disagreement into silence. - Do a sensitivity pass: rerun with alternate weights (e.g., "what if predictability matters twice as much?"). An option that wins under every reasonable weight set is robust; one that only wins under one specific set is fragile, and you should say so. - Score ranges instead of points where you're unsure ("3-4"), and note which criteria are the uncertain ones. - Present the output as "Kanban wins IF adaptability is your top priority; Waterfall wins IF predictability is" — not "Kanban is better." The matrix's real product is the forced discussion about weights and scores. The number at the bottom is just the receipt.