Outcome Bias: This is What You Miss When You Judge Decisions by Their Outcomes

When Niki walked into her boss's office, she had the letters spread out across her desk. Niki's boss was waiting for her. “You'll never be successful if you behave like this,” her boss said. “It's only a 8 rupee stamp (~$0.10), but this inattention to detail on a larger scale is going to sink you.”

Niki had put the wrong stamps on envelopes. She made a calculated decision because they were out of the regular stamps, and the letters had to go out that day. They were healthcare notices, and it would have been illegal to miss the deadline. Her boss didn't see it that way.

“You may not have a job here much longer. Certainly not a full-time one when you graduate.”

Now, consider this thought experiment: one million monkeys trade on the stock market. They buy and sell stocks completely at random everyday.

After one week, about half of the monkeys make a profit and the other half a loss. The ones that made a profit stays; the ones that made a loss are eliminated and sent back to the zoo. In the second week, one half of the monkeys will still be riding high, while the other half will have made a loss and get sent back. And so on…

After ten weeks, about one thousand monkeys will be left — those who have always invested their money well. And after twenty weeks, just one monkey will remain — this monkey consistently invested in the right stocks, and is now a billionaire.

The media pounces on this special animal, trying to understand its success principles. Perhaps he eats more bananas than the others. Perhaps he sits in one corner of the cage and meditates. Or maybe he interacts with the children more than others, or he takes long, reflective pauses while swings across branches. A book called “How to be a success monkey” is written depicting his rags to riches life.

He must have some recipe for success. How else could he perform so brilliantly and so consistently?

Both the stories illustrate The Outcome Bias.

You tend to evaluate decisions based on the result rather than on the decision process.

Take a moment to imagine your best decision in the last 6 months. Now take a moment to imagine your worst decision. It's very likely that your best decision preceded a good result and the worst decision preceded a bad result.

The outcome bias happens because of “resulting”. It's a routine thinking pattern that bedevils you. Drawing an overly tight relationship between results and decision quality affects your decisions every day — potentially with far-reaching, catastrophic consequences.

But you are not alone. Leaders and managers tend to do the same everyday. I have yet to come across someone who doesn't identify their best and worst results rather than their best and worst decisions.

Rajeev runs a product company. He recently had to fire his CMO. He feels this is his worst decision in the last couple of years. This has hurt his company the most.

“Since we fired him, the search for a replacement has been awful. We haven't had anybody come in who actually turns out to be as good as he was. Sales are falling. Employee motivation is record low,” he laments.

That clearly sounds like a disastrous result, but it would be good to understand how he went about making the decision. Other than that it didn't work out, what else was wrong with it?

“We looked at our direct competitors and comparable companies, and concluded we weren't performing up to their level. We thought it was probably a leadership issue. We could definitely perform and grow at that level if we had better leadership.”

“We started working with our CMO to identify his skill gaps. We even hired an executive coach to work with him on improving his leadership skills, and get his major weaknesses identified.”

“But it still failed to produce improved performance. We debated splitting his responsibilities and have him focus on his strengths, and moving other responsibilities to a different executive.”

“But this idea was rejected, concluding that the CMO's morale would suffer, employees would likely perceive it as a vote of no confidence, and it would put extra financial pressure on the company to split a position we believe one person could fill.”

It sounded like Rajeev had a reasonable basis for believing they would find someone better. The company had gone through a thoughtful process and made a decision that was reasonable given what they knew at the time.

It sounded like a bad result, not a bad decision. The imperfect relationship between results and decision quality devastated Rajeev and adversely affected his subsequent decisions regarding the company.

Rajeev had identified the decision as a mistake solely because it didn't work out. He obviously felt a lot of anguish and regret because of the decision. He stated very clearly that he thought he should have known that the decision to fire the president would turn out badly.

He was not only resulting but also succumbing to its companion — Hindsight Bias. Hindsight bias is the tendency —after an outcome is known — to see the outcome as having been inevitable. When you say, “I should have known that this would happen,” or, “I should have seen it coming,” you are succumbing to hindsight bias.

I'm yet to come across a founder who acknowledges a bad decision where she got lucky with the result, or identifies a well-reasoned decision that didn't pan out. You generally link results with decisions even though it is easy to point out indisputable examples where the relationship between decisions and results isn't so perfectly correlated.

The firing of the CMO was an example where the outcome didn't pan out as expected. Now take the opposite case: a drunk driver. When he leaves the pub with a skin-full of alcohol in him, he probably knows it's a bad idea to drive drunk — particularly the first time that he does so.

However, if his drunk driving does not result in an accident, then he is more likely to drive drunk in future. Unlike the firing of the CMO, the outcome of this action had no negative consequences — so to him there's nothing wrong with repeating the action.

Yet, as you know very well, drunk driving is a bad idea. Any decision making process that leads to drunk driving is flawed. Proper evaluation of the decision making process is more likely to deter the drunk driver in the future than focusing on the outcome.

No sober person thinks getting home safely after driving drunk reflects a good decision or good driving ability. Changing future decisions based on that lucky result is dangerous and unheard of — unless you are reasoning this out while drunk and obviously deluding yourself.

Yet this is exactly what happened to Rajeev. He changed his behaviour based on the quality of the result rather than the quality of the decision-making process. He decided he drove better when he was drunk.

You see this a lot with poor managers. A manager who makes a decision based on his “gut instinct” — one who goes in the opposite direction because his gut says so when his team is strongly advising him in one direction — is going to consider his gut feeling to be reliable if he gets a positive outcome from it. These managers suffer from Confirmation Bias — their false belief on their gut is confirmed (thus boosting their overconfidence) when the outcome is luckily in their gut's favour. Gut feeling is synonymous to System 1 thinking — not the best for executive decision making. These managers are pretty much like the drunk drivers.

If you think there are a lot of “drunk driver” managers driving your company bus who got lucky so far, and are waiting to meet his first accident, it's better to identify them and get them off the wheel.

Outcome bias is much more likely in negative events. Consider the following scenario: a real estate agent doesn't disclose to the customers the high probability of the basement of the house flooding. This risk exists at the time of his decision not to tell his customers, regardless of whether there actually is a flooding later or not.

However, in reality the agent is likely to be viewed much more negatively if the basement is actually flooded. While this is emotionally understandable, purely logically it should not make a difference. This decision should be viewed negatively in both cases.

Outcome bias is your tendency to judge a decision by its eventual outcome instead of judging it based on the quality of the decision at the time it was made.

This is costly, especially to organisations. It causes employees and leaders to be blamed for negative outcomes even when they had good intentions and used a thoughtful decision-making process, considering all the information that should be taken into account. This is what happened with Niki in the first example.

Organisations and their leaders can benefit from understanding how to help individuals look beyond end results. Francesca Gino, Ovul Sezer and Max Bazerman of Harvard, with Ting Zhang of Columbia decided to investigate potential ways to eliminate (or at least reduce) the outcome bias.

They focused on two possible solutions: evaluating multiple scenarios simultaneously rather than in sequence in order to best evaluate the quality of a person's decision, and highlighting the role of intentions during the evaluation process.

In one study, for instance, participants evaluated a physician who could choose between two equally effective drugs to treat a patient. Participants also learned that, although the drugs were equally effective in clinical tests, one was cheaper for the patient and the other would generate more revenue for the physician. Participants in the separate-evaluation condition evaluated one of the following two conditions:

1. A physician prescribed the cheaper drug to save the patient money. Despite these good intentions, the patient suffered from adverse side effects and spent the night at the hospital.
2. A physician prescribed the more expensive drug to generate more revenue for the clinic. Although the physician had selfish intentions, the patient made a full recovery without side effects or hospitalisation.

Participants in the joint-evaluation condition evaluated both physicians.

They came up with the following findings after a series of studies:

1. Common biases are not necessarily reduced or eliminated when people evaluate courses of actions at the same time (simultaneously) rather than one after the other (separately). Joint evaluators are more likely to neglect intentions and overweight outcomes as compared to separate evaluators. Joint evaluation exacerbates, rather than lessens, the effect of the outcome bias.
2. You are most likely to reward the selfish partner you had been assigned to work with because the outcome of this person's decision was good, but you would not reward the well-intentioned partner whose decision led to a poor outcome. And when you had to decide whom to work with on a follow-up task, you are most likely to prefer the selfish partner over the well-intentioned one. For example, you would most likely prefer the physician who prescribed the expensive drugs.
3. If you make judgments about decision makers' choices before the result of those outcomes, you are likely to reduce the outcome bias under joint-evaluation contexts but not in separate-evaluation contexts. For example, if you hadn't known what effect the cheap and expensive drugs had on the patients, you are likely to prefer the well-intentioned physician.

The lesson: Requiring individuals to make judgments about people's decision-making process before the outcomes have been achieved is an effective strategy to reduce the outcome bias in contexts in which scenarios or decisions are evaluated simultaneously rather than one after the other.

For instance, when deciding whom to hire or promote, or what raise to give to employees, you generally evaluate multiple people at the same time. In these situations, making decisions about them prior to looking at whether their decisions led to good or bad outcomes would assure you an unbiased process.

It is easy to celebrate or lament a particular decision or action based on how it turns out. But it is important to remember that the process that led to the outcome, including the decision maker's initial intentions, deserves to be taken into account in evaluating the results. It's important for you to be truly critical in the way you evaluate yours and others actions. Instead of focusing on outcomes, you need to focus on the process as a whole:

1. What led you to take the decision?
2. What information did you or didn't you have at that point?
3. Was there a better process that you could have followed to make the decision?
4. Could you have consulted other people?
5. Could you have obtained more data?
6. Was there any need to make the decision when you had made it?

In conclusion: Never judge a decision purely by its result, especially when randomness and “external factors” play a role. A bad result does not automatically indicate a bad decision and vice versa.

So rather than tearing your hair out about a wrong decision, or applauding yourself for one that may have only coincidentally led to success, remember why you chose what you did.

Were your reasons rational and understandable? Then you would do well to stick with that method, even if you didn't strike it lucky last time.