The Decision Journal: Record Today, Decide Better Tomorrow

Why do owners learn so little from their own decisions?

An owner makes dozens of decisions a year that cost or bring money: hires, prices, investments, new customers, new markets. You would think he gets better at it every year. That is only partly true. He becomes more experienced, but not necessarily more accurate, because he rarely learns how good his judgment actually was before the decision.

The reason is simple: the memory of what you expected at the time changes as soon as the outcome is known. If you wrote nothing down, you compare the outcome with an expectation smoothed over after the fact. The central question, then: how do you record decisions so that you can later test your judgment and not just your luck?

To answer it, I first clarify two terms, hindsight bias and calibration, and show what studies say about both. Then come the structure of a journal, a worked example of the review, and the question of where the method reaches its limits. The guide builds on the essay Deciding under uncertainty, which introduced the three-sentence note.

Hindsight bias: why we always knew it

Hindsight bias describes the tendency to consider an outcome more predictable after the fact than it was. The psychologist Baruch Fischhoff studied it experimentally in the 1970s. It has since been confirmed in many fields, from court rulings to elections.

Kara-Yakoubian and Spaniol (2025) provide an example. They asked 272 younger and older adults to predict the electoral college votes before the 2020 U.S. presidential election and, about three weeks after the result, to recall their prediction. The memories had shifted toward the actual result, and feelings about the outcome influenced how strongly. Bryant and Guilbault (2002) found a similar pattern among students who assessed the outcome of the impeachment trial of President Clinton at several points in time.

For a business this means: if after a failed project you tell yourself you actually saw it coming, you are probably wrong. And if after a success you believe you foresaw it with certainty, you are probably wrong too. Both prevent learning. In the first case you believe you merely failed to listen to yourself. In the second you overestimate your own judgment.

The model: calibration

The goal of a decision journal is not to always be right. Under uncertainty that is impossible. The goal is calibration. A judgment is well calibrated when the stated confidence matches the actual hit rate. If across many decisions you say you are 70 percent sure, you should be right in about seven out of ten cases. If you are right in only five out of ten, you are overconfident. If you are right in nine out of ten, you are overly cautious.

How calibration can be measured and improved was studied by Moore, Swift, Minster, and colleagues (2017) in an unusual data set. It comes from a forecasting competition run by a U.S. research agency in which thousands of participants predicted the outcome of hundreds of political events over three years. Performance was measured with the Brier score, a measure that takes into account both accuracy and the stated probability.

The results are encouraging. Overall, participants' confidence matched their accuracy fairly well, and both rose together as more information came in. A small overconfidence of about 3 percent remained, especially on the most confident forecasts. Brief training reduced it, teamwork improved accuracy, and both together brought overconfidence down to about 1 percent. Part of the training was to take the outside view first, that is, to ask how often something comparable has happened under similar circumstances.

The authors name the limits themselves: participants were volunteers, and the questions had clear answers and fixed deadlines. Business decisions are rarer, more complex, and harder to measure. Kelly and Mandel (2024) also showed with 70 intelligence analysts that training worked differently depending on the task and could even reinforce existing underconfidence. The core still holds: calibration can be learned, but only with records and feedback.

What belongs in a decision journal?

The journal is the extended form of the three-sentence note. Its three sentences remain the core: expectation, reasoning, mind-changer, plus the date for review. Four fields are added that make the review possible.

The "state of mind" field seems unusual but is useful. After a year it often shows that certain decisions turn out worse in certain states, for example late at night or right after a setback. How rest and judgment are connected is described in the essay The owner's workload.

A decision journal does not protect you from mistakes. It protects you from drawing the wrong conclusions from luck and bad luck.

How to keep the journal: a four-step routine

1. Select

   Not every decision belongs in the journal. Take the ones that cost more than one employee's monthly revenue, are hard to reverse, or keep you up at night. For most businesses that is 15 to 30 a year.

2. Write beforehand

   Fill in the five before-fields before you decide, not afterward. Ten minutes are enough. Do not forget the date.

3. Schedule the review

   Set the date in your calendar when the outcome will be measurable. Without that date the journal remains a diary.

4. Review

   On the date, read your old entry first and only then the numbers. Place the decision in the grid of process and outcome and record one lesson in one sentence.

The grid of process and outcome comes from the article on deciding under uncertainty. It distinguishes deserved success, luck, bad luck, and deserved failure. The author Annie Duke calls the mistake of looking only at the outcome resulting. The journal is the tool that prevents this mistake in everyday work, because it records the process in writing while the outcome is still unknown.

How to check your calibration after a year

After twelve months you have enough entries to see a pattern. Sort them by the stated probability into rough groups and count how often the expectation came true. The following worked example shows what a typical result might look like.

In this example the owner has no problem with uncertain decisions, but he does with the decisions he was very sure about. Two practical rules follow. First: for anything you consider very certain, get a second opinion. Second: in the reasoning and outside-view fields, ask more honestly how often comparable things actually work.

If there are several decision-makers in the business, the review can be done together, for example in the annual strategy check. For debriefing individual bad decisions, the learning round from Learning from bad decisions is suitable. For projects, Project closure and lessons learned describes the right procedure.

Where the decision journal reaches its limits

There are two objections worth taking seriously. The first is that the journal is too much effort and turns every decision into an administrative act. That is true if you keep it for every small thing. For reversible decisions with low stakes, the article on deciding under uncertainty already recommends deciding immediately and without effort. The journal is meant only for the few decisions where a mistake becomes expensive.

The second objection carries more weight: business decisions are too rare and too varied to learn anything statistically from them. Twenty entries a year are not a sample like in a forecasting competition. My assessment: for precise measurement that is indeed not enough. For recognizing rough patterns it is, for example that you regularly set time frames too short or are too optimistic about hiring decisions. And writing things down before the decision already improves it, because it forces you to name alternatives and mind-changers.

Where reliable numbers exist, they take precedence. Which few key figures are enough for this in a business is shown in the essay Data-driven decisions. The journal supplements them for the cases in which the numbers are not sufficient.

How to get started

• Create a document or notebook titled "Decision journal" and copy the seven fields from the table.
• Write the first entry for a decision coming up in the next two weeks, with a probability in percent.
• Put the review date in your calendar, on the day the outcome will be measurable.
• Set yourself a date twelve months from now for the first calibration check.

The first entry feels unfamiliar, especially the number in percent. That is intended. If you have to state a probability, you quickly notice how sure you really are.

Conclusion: how do you record decisions so that you learn from them?

The guiding question was how to record decisions so that you can later test your own judgment and not just your luck. The answer: by stating in writing before the decision what you expect, how sure you are, and what would change your mind, and by comparing expectation and outcome on a fixed date. Hindsight bias makes exactly this comparison impossible without a record.

The decision journal is not a tool for perfect decisions but one for honest feedback. Research on calibration shows that judgments become more accurate with practice and feedback. An owner who records twenty decisions a year this way knows more about his own judgment after one year than after ten years of experience without records.