Regret Minimisation Framework: How to Live With Minimal Regret

I went out for dinner last night. I had to decide where to eat, what to wear, how to go there, etc. All of us have to make such choices on a daily basis. These are little choices. They have very little downside even if I make the wrong choice. But there are big choices as well.

When we choose which job offer to take, who to hire, or what city to live in, the consequence are high, and hence regret looms large. Whenever we are presented with a set of options—all of them good—we can easily torture ourselves with the consequences of making the wrong choices.

Regret is the comparison between what would have been the best course of action in hindsight versus what course of action we actually took. It’s the difference between the payoff that could have been obtained by pulling the best strategy, and the payoff actually obtained by following a particular strategy.

Before I get deeper, let me break it to you. No matter how well you bat, you can’t score a century every single time. And no matter how deep you are in your thought, quick in your wit, and prudent in you action, you cannot pull off the best strategy every single time. Hence, you would often meet with regret—both little and huge—during retrospective analysis.

It is virtually impossible to live a life without regret, yes, but if we use the Regret Minimisation Framework (RMF) the next best thing is indeed achievable: living a life with minimal regret.

RMF forces you to think beyond the moment, past all fears and doubts, and consider your decision among the grand scheme of things. Before he decided to start Amazon, Jeff Bezos had a secure and well-paid position at the investment company D. E. Shaw & Co. in New York. Starting an online bookstore in Seattle was going to be a big leap—something that his boss advised him to think about carefully. Bezos recollects:

“The framework I found, which made the decision incredibly easy, was what I called—which only a nerd would call—a Regret Minimisation Framework. So I wanted to project myself forward to age 80 and say, ‘Okay, now I’m looking back on my life. I want to have minimised the number of regrets I have.’ I knew that when I was 80 I was not going to regret having tried this. I was not going to regret trying to participate in this thing called the Internet that I thought was going to be a really big deal. I knew that if I failed I wouldn’t regret that, but I knew the one thing I might regret is not ever having tried. I knew that that would haunt me every day, and so, when I thought about it that way it was an incredibly easy decision.”

Turns out, fear of regret can actually be very motivating, especially when you have to make tough decisions. Regret Minimisation Framework forces you to think beyond the moment, past all the fears and doubts that you may be have.

In his famous book Algorithms to Live By, Brian Christian writes about the postulates of the Regret-Minimising Algorithm (the mathematical counterpart of our Regret Minimisation Framework), devised by mathematicians Herbert Robbins and Tze Leung Lai:

“First, assuming you’re not omniscient, your total amount of regret will probably never stop increasing, even if you pick the best possible strategy—because even the best strategy isn’t perfect every time. Second, regret will increase at a slower rate if you pick the best strategy than if you pick others; what’s more, with a good strategy regret’s rate of growth will go down over time, as you learn more about the problem and are able to make better choices. Third, and most specifically, the minimum possible regret—again assuming non-omniscience—is regret that increases at a logarithmic rate with every pull of the handle.”

Logarithmically increasing regret means that we’ll make as many mistakes in our first year as in the rest of the decade combined. The first decade’s mistakes, in turn, are as many as we’ll make for the rest of the century. This essentially means that you should try new things, experiment as much as possible, but don’t repeat the same mistakes. Over time, as mistakes reduce, so would your regret.

Christian buttons this up by writing:

“In general we can’t realistically expect someday to never have any more regrets. But if we’re following a regret-minimising algorithm, every year we can expect to have fewer new regrets than we did the year before.”

RMF has close ties with building your Circle of Competence. When you start a new business, you are very likely to make a lot of mistakes. But as times passes, and as you become an expert in the field, your develop your circle of competence, and as long as you are not stupid to repeat mistakes, you are in good hands. It doesn’t guarantee your success, but it does increase your odds.

“To try and fail is at least to learn; to fail to try is to suffer the inestimable loss of what might have been.” — Chester Barnard

A corollary of RMF is optimism. It helps to be optimistic, especially in the face of uncertainty. When you are starting out with your exploration, when most of the terrain is uncharted, assume the best as long as you don’t find any evidence to the contrary. Clearly, exploration would lead to mistakes, and that’s OK at the start.

When starting a business, assume it would be great. In everyday life, you should be be excited to meet new people, and try new things. When you do, assume the best about them. In the long run, optimism is the best prevention for regret.

Christian writes:

“Optimism, they show, can be perfectly rational. By focusing on the best that an option could be, given the evidence obtained so far, these algorithms give a boost to possibilities we know less about. As a consequence, they naturally inject a dose of exploration into the decision-making process, leaping at new options with enthusiasm because any one of them could be the next big thing. The same principle has been used, for instance, by MIT’s Leslie Kaelbling, who builds “optimistic robots” that explore the space around them by boosting the value of uncharted terrain. And it clearly has implications for human lives as well.”

Having said that, there’s a huge difference between practical optimism and irrational optimism. When starting out a business, it makes sense to be optimistic about it. It keeps the moral of others along with yours high. But when you find contrary evidence, don’t ignore them.

Also, plan for all contingencies. Bezos had the option from his boss (D.E. Shaw himself) to get back to the firm after trying his stint at building Amazon. He had a safety cushion. If you only have a plan, and if it fails you are likely to become a pauper, it’s not such a good plan after all—as the risk of regret is tremendously high. Be an optimist, just not a blind one.