What is a decision journal?

A decision journal records what you decided, why, what you expected to happen and how confident you were, written before the result is known. It exists because memory reorganises itself around outcomes, so a review conducted afterwards is mostly a review of a reconstruction.

The problem it solves#

Hindsight bias is among the best established findings in the literature on judgement. Once an outcome is known, people reliably report having expected it, and they report it sincerely. The memory of what they thought has been rewritten rather than suppressed. This means that the ordinary professional practice of looking back at a decision and asking what we were thinking cannot recover what anybody was thinking.

It also means that experience in a setting without records teaches a flattering lesson. Decisions that worked are remembered as well reasoned and decisions that failed as obviously flawed, which builds confidence in a decision process that has never actually been examined. That is one of the mechanisms that makes professional work a wicked learning environment.

What goes in it#

• The decision and the date. Including the options not taken, which are the part memory disposes of first.
• The reasoning. What you were weighing, and what mattered most.
• The expected outcome. Stated concretely enough to be wrong.
• A confidence figure. A number, because numbers can be scored and adjectives cannot.
• The falsifier. What would tell you, early, that this was going the wrong way.

The last two are the entries people skip and the two that make the record worth keeping. Without a number there is no calibration to check. Without a falsifier the review has no early point of comparison and has to wait for the final result, which in most professional work arrives too late or not at all.

What the evidence supports#

There is no trial of decision journals. Nobody has measured whether keeping one improves decisions, and any page claiming otherwise is overstating. The support is indirect and reasonably strong in its parts. Hindsight bias is well established. Calibration improves when confidence judgements are scored against resolved outcomes, which requires that the confidence was written down. And the feedback literature finds that reviewing how a judgement was formed beats being told whether it was right, which requires a record of how it was formed. A journal is the precondition for two practices that do have evidence, rather than an intervention with evidence of its own.

Making it survive contact with a working week#

The failure mode is not disbelief, it is volume. A journal that covers every decision is abandoned within a month. The version that survives covers a defined class: decisions that would be expensive to get wrong, or that will not be revisited, or where you noticed yourself feeling certain. Five entries a month that are honest beat fifty that are dutiful. The organisational form of the same idea, kept for each class of decision rather than for each decision, is the Decision Quality Protocol.

The AI entry#

Where a tool was involved, three extra lines are worth more than the rest of the entry: what it proposed, what you changed, and why. That record is what makes it possible to tell later whether you were exercising oversight or approving. It is also the only way to notice the slow version of the problem, where the proportion you change falls quietly over a year and nobody has a baseline to compare against. The behaviour that record is testing for is described at appropriate reliance.

What this does not establish#

That keeping a journal makes anyone a better decision maker. It makes a later review possible and honest, which is a precondition for learning rather than learning itself, and a journal that is written and never read produces nothing at all.

Key sources

• Balzer, W. K., Doherty, M. E. and O'Connor, R. (1989). Effects of cognitive feedback on performance. Psychological Bulletin, 106(3).
• Lichtenstein, S. and Fischhoff, B. (1980). Training for calibration. Organizational Behavior and Human Performance, 26(2).

Related SuperSkills research#

• What is an after action review
• What is calibration training
• How is judgement trained