Thinking in Bets is Product Management

Annie Duke, Author of Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts, and former professional poker player, shares her approach to making decisions in an uncertain environment, overcoming cognitive biases that lead to poor decision-making, and making small bets to learn.

In poker, you have to make high-stakes decisions with limited information. As a result, there’s only a loose relationship between outcomes and decision quality.

Annie mastered decision-making in this environment. She won a World Series of Poker gold bracelet, the World Series of Poker Tournament of Champions, and the NBC National Heads-Up Championship, and earned more than $4 million in tournament winnings.

It turns out, making decisions in poker has a lot in common with making decisions in product management. This is because "every single decision that we make is really a bet." Annie explains that betting is actually about determining the possible futures and what are the chances of those futures:

When you choose to make a particular bet, you're forgoing all the other decisions that you could make at that moment because you're guessing that the set of futures, the way that things will turn out given that decision that you make will have a better return than all the other decisions that you might make.

In product management, it’s hard to predict what products and features will be successful. If you’re creating a new experience, you don’t have historical usage data to rely on, and customer needs and market dynamics are always changing. Once you do get results though, it is crucial that you don't just sit and bask in the glory of your win. Otherwise, "you won't actually go back and figure out what you can learn from that win and what was sort of just due to luck."

To make smarter decisions, Annie recommends thinking in terms of the probabilities of potential outcomes, getting input from others to counteract our natural cognitive biases, and making smaller bets to gather information and improve your ability to make bets in the future:

We want to become really good gatherers of information, but in order to do that, we have to have a very open-minded stance to people and opinions that disagree with us, number one. And number two, we need to be recognizing that we aren't certain, and not have certainty as the goal.

To this end, Annie stresses the need to change the tenor of boardroom conversations and other meetings:

Instead of saying, 'This is what I believe. I know it's true,' it's actually much better to say, 'Here are the possibilities, and here's the decision that I have to make.'

This process of listening has other important ramifications if the product does actually get shipped. "Because they've told you what the bumps in the road might be," Annie says, "when those bumps do occur, you're not so surprised by them. You're not so reactive as an organization." This type of thinking is what leads Annie to relate the importance of constant testing and iteration:

You should be making these little low cost stabs at things all the time because that information that you gather by essentially testing, turns something that could be counterfactual into an actual testable hypothesis. Instead of saying well, what if we had launched this product, actually do it and see what happens, as long as it's low cost, because the return on investment you get on that is so high, that making these little stabs can really make sense. You're gathering information in order to be able to win a lot more money later.

You’ll learn a lot from this episode about decision-making, forecasting, and resource allocation.

Here are the highlights:

• How Annie’s success making decisions at the poker table translated into making decisions in the boardroom (8:30)
• How Annie makes business decisions without having all the facts (13:15)
• Annie’s approach to gathering information to make a decision about a new product or feature (24:40)