A Conversation with Annie Duke: Improving Your Decision Skills

Annie Duke was one of the world’s top poker players for two decades, earning over $4 million in tournament winnings. Annie is the author of Thinking in Bets, How to Decide, Quit, and the co-founder of the Alliance for Decision Education. She also runs an excellent decision-making course available on Maven (I know because i’ve taken it!).

I had the privilege of talking to Annie about what it actually means to get better at making decisions.

You can listen to the whole conversation above or read the transcript below.

Here’s what you’ll learn if you read or listen:

• Why no decision is truly a “one-off”.

• How to practice decision-making (fantasy sports, poker, and why the feedback loop matters more than the activity).

• Can you think probabilistically even in situations of radical uncertainty?

• Why decision journaling usually fails.

• The one thing Annie recommends people do to start taking responsibility for the quality of their decisions.

---

I recommend you listen to the conversation, but here is a transcript, with minor edits for clarity, brevity, and flow.

The Only Two Things That Determine How Your Life Turns Out

Stephen: Annie, welcome.

Annie: Thanks for having me. Excited to be here.

Stephen: Good to see you again. I think a good place to start might actually be how I discovered your work and why it resonated with me. I originally did my bachelor’s and my master’s in judgment and decision making, but in the moral realm — all about moral decision making. It had a big focus on Jonathan Haidt’s social intuitionist model. I was basically trying to answer the question of, if there was such a thing as moral expertise, what might that look like? Then, fast forward to 2019, when I was searching about different kinds of thinking styles, I came across Thinking in Bets. What I really took away from it was that it connected the dots for me on decision making being its own learnable skill. I’m wondering if you could talk to me a little bit about improving decision skills and why you’re so confident that’s actually something we can do.

Annie: So let me just start with this. There are only two things that determine how your life turns out: luck, which you have no control over, and the decisions that you make. What luck sets into place is the distribution of the outcomes that are available to you at the beginning of your life. If I had been born in 1600, that’s a very different set of outcomes than being born when I was, particularly as a woman. Being born in America versus a third world country. Who are your parents? How much do they care about education? Are you tall? That’s going to determine your probability of being in the NBA. You don’t have any control over that stuff. So that leaves the decisions that you make. The decisions you make are actually affecting the probability that you observe certain outcomes in your life. If you’re good at decision making, it’s going to increase the probability that you end up with better outcomes than you otherwise would. It’s not going to guarantee it, but it’s going to push you toward the right tail. In terms of whether this is a skill that can be taught — the history of the science has been to have this very big focus on the ways that our decision making goes wrong. For people who’ve read Thinking, Fast and Slow, that’s really cataloguing a lot of the types of errors in thinking we can succumb to. Cognitive bias, heuristics, sunk cost fallacy, overconfidence, confirmation bias. Those ideas were actually quite controversial in the seventies. Starting with Kahneman and Tversky, Richard Thaler, and others, they really showed that these errors are real. That’s where the focus was for a long time. What’s shifted more recently is people asking: okay, but what do you do about it? You can’t just know about it and think it’s going to go away. Knowing you’re subject to confirmation bias alone is not enough. People like Phil Tetlock and Barb Mellers, my own work, Katie Milkman, have been thinking about how we actually create processes and structures that reduce the impact of these errors. And it turns out there are a lot of scientifically backed interventions that can improve decision making. That’s a lot of the more recent work from the last decade, decade and a half. I just really believe in getting that message out and giving people the tools to become better decision makers.

From Pessimism to Practice: How Decision Science Changed

Stephen: I think that’s also one of my focuses here — stemming from what I studied, I think what we do and how to do it is essentially the fundamental human problem. That’s why I push it as far as saying we have a responsibility to ourselves to work on it, because it’s the only thing we have control over that can determine everything else. I like that you’ve picked up on this swing that’s happened in the field, from what I see as quite a pessimistic outlook — it’s all about what goes wrong, we’re fallible, and a lot of that material makes people feel quite nihilistic. But now we’ve swung more toward forward-looking things we can actually do to build structures around making better decisions.

Annie: Yeah. What was happening, particularly in economics for a long time, was this idea that we act in our own best self-interest — rational actor theory. When Kahneman, Tversky, and then Thaler came along, they were very poo-pooed. So much of the focus was on how systematic and predictable these errors are, rather than how to fix them, because there was so much convincing that needed to be done. I think your point is well taken. I would add something: Kahneman is famously pessimistic — famously so. When you’re diving into how pervasive and durable these errors are, you can become very nihilistic. But for lay people interacting with this material, I actually find the opposite — they think that because they know about it, they’re fine and won’t do it. They can list off sunk cost fallacy. If I buy a stock at 40 and it’s trading at 30, I should be neutral to whether I’m in gains or losses — the fundamentals and forward-looking ROI are what matter. But people say “yeah, but I know about that, so I don’t do it.” The scientists were saying these things are so durable and hardwired that we’re just going to make errors. The general public hearing about the material had the opposite response: now I know, that’s a competitive advantage, I’m not going to do it anymore. That’s where the convergence comes in. The research, which is very pessimistic, has become more optimistic. And people who were way too optimistic need to become more pessimistic — to realise knowing about it is not enough, and they need to be using these structures in their own lives.

How to Get Your Reps In

Stephen: And one thing I find quite interesting when that reality hits people — and we talk about this as a skill — is what does practice look like? I’m curious to talk about how you get your reps in. When I did your course last year, we spoke about how people were trying to integrate this into their daily lives. For me, I chose fantasy sports about seven or eight years ago, solely as a good way to get reps in. It’s got enough hidden information, a good blend of luck and skill. That’s been incredibly helpful for disciplining a whole range of habits. I’m wondering if you could talk a little bit about that. What I’m particularly interested in is how do we do it well? With decision journaling, it didn’t work for me because I needed something tangible where I wasn’t so tempted to lie to myself. It’s easy to write it down and say “yeah, I would take that bet” — but when there’s nothing on the line, no skin in the game, I really struggled with that. Fantasy sports worked because you’ve got to make a decision, the deadline comes, it’s out there, it’s competitive. What are your thoughts on good ways to get your reps in?

Annie: I’m not a big fan of decision journaling. I think it’s kind of mushy. Some of the things I teach might look like journaling, but it’s actually not — it’s creating structure around your decision making and making sure you’re quantifying your opinions pretty precisely, in a way that allows you to close your feedback loops. When people talk about journaling, they’ll say “I did this because blah blah blah” — I don’t really know what that means, and I particularly don’t know because, as you know from fantasy sports, what does one outcome mean? You go back to your journal and you’re going to say you got unlucky if it’s bad and exactly right if it’s good. I want a rubric. I want forecasts. I want things on a Likert scale — be precise — and then do that over lots of decisions. Things you’ll see in a decision journal: “I think there’s a really good chance that...” — what does that mean? Those are words describing some probability and you haven’t said what you think the probability is. Obviously I came from poker, so I was repping this all the time in that environment. To get your reps in its fullest form, you need an environment with hidden information and a lot of short-term influence of luck. In fantasy sports or poker, there isn’t a lot of long-term influence of luck, but in the short run there is. If I flip a coin, twenty-five percent of the time I’ll get two heads in a row. That doesn’t mean heads is a hundred percent. Twelve and a half percent of the time I’ll get three heads in a row. Six and a quarter percent of the time, four heads in a row — that’s actually quite often. But if I flip it ten thousand times, I’ve taken the luck element out and I’ll see it’s fifty-fifty. Fantasy sports, poker, options trading — similar in that sense. To become a good decision maker, you need both hidden information and a strong influence of short-term luck. Chess will help you think in decision trees — if-thens. If I move here, here are the possible moves they could make, and I can think about the probability of each. That’s valuable. But not a super strong influence of luck in the short run, and not much hidden information. If you really want it: play cards, do fantasy sports. But also — and this is the interesting thing — most things we decide about in life are actually very similar to that. For important decisions, start structuring your decisions in a way that allows you to close the feedback loop and understand what happened. That takes more than a journal.

Feeling Twenty-Five Percent in Your Bones

Stephen: The way I always think about it — and what I’ve found from using it as a decision-making exercise — is there really isn’t a substitute for having that kind of experience, particularly for understanding short-term versus long-term. When I first started playing seven or eight years ago, I was so caught up in the near-term information every single week. The best way I can describe how I play now is: in the first week of the season, I’m simultaneously in the last week. There’s almost no difference to me now, because I’ve had that experience of knowing what information reveals itself over the short term and the long term.

Annie: The way I describe it with poker — when people ask me what poker really taught me — is: I can feel twenty-five percent in my bones. If I’m a three-to-one favourite, seventy-five percent of the time I’ll have the good outcome and twenty-five percent of the time the bad one. That’s actually quite noisy. It means twenty-five percent of the time I will observe a bad outcome, but I don’t know if I will observe it on this occasion. What happens is when people observe that twenty-five percent — there’s a bad card, they lose the pot — they just think they got so unlucky. And when the seventy-five percent occurs, they think that was just supposed to happen. No — it was supposed to happen seventy-five percent of the time. When you play that game as long as I played it, and you get into all these situations, you start to really feel the probability differently. The world is probabilistic, but everything settles to one or zero. On a poker hand, I can only win or lose. You can watch an NFL game and track a team’s win probability toggling around — 35%, then they score and jump to 52% — but in the end, they either win or don’t. If I get in my car and I’m 0.5% likely to have an accident, on that drive I either have one or I don’t. The clearest example in everyday life is a weather forecast saying there’s a 75% chance of rain. It either rains or it doesn’t. And we all know that when it rains we say the forecast was right, and when it doesn’t rain we say it was wrong. No — on that particular day it was going to settle to one or zero. This is a really big problem for how we interpret the world and our decisions. Poker solved that for me. Repping that over and over got me out of the trap of putting too much weight on whether it happens or doesn’t on a given try.

Stephen: Yes. And it’s getting those outcome reps in high volume, which is the difficulty when we think we can reliably practise decision-making skills just as we go about our lives generally. You only get that feedback on moving house maybe a couple of times in your life. The feedback loops aren’t there.

The One-Off Fallacy

Annie: Yeah. Although it’s interesting — people think, well, I’m only going to get married once, or hopefully once. Or how many times am I actually going to move house? Does this really apply? How could I know what the probability is? How could I close that feedback loop? And then I talk to people in early-stage venture capital and they’ll say, I’m not even going to find out how it turns out until ten or fifteen years from now. The feedback loop is just too long. And they’ll cite power law — you might have a portfolio with forty companies, three of them are going to be the fund drivers. How could you tell whether the other ones were poor decisions? I don’t accept any of that. On moving house: other people have moved house before. Go look at them. You’re not the only person who has ever moved house, and you’re not that different from other people. You can always find other reps that will inform the decision you make. Go find data. If you’re going to buy a house and live there forever — go look at the average length of time someone owns a house. That informs all sorts of decisions you make around it. Know that your income is going to increase and you’re willing to pay more than you probably should on your mortgage — go look up the data on how that works out for people. There’s all sorts of things you can do, even for a one-off, if you go look at the base rates. Second: there is no decision that’s really isolated in the arc of your life. You make thousands of decisions every day, and in terms of really consequential decisions, you make a lot of those — what college, who to marry, where to live, what house to buy. You should think about it as a portfolio of decisions. Going to venture capital: the feedback loop isn’t ten to fifteen years. What are you even talking about? You invest and then all sorts of things happen — does it raise another round, is revenue growing, are they making key hires, what does churn look like? Why are you accepting that the feedback loop is that long? That’s true for a house too. Do you like the neighbourhood? Were there unexpected repairs? Is it financially taxing? You know all of those things in between. Treat it that way. And in terms of luck in the short run — that’s why you have portfolio theory, to smooth it out. Treat your decisions the same way. On average, is the way I think about these things predicting better outcomes or worse outcomes? Good decision making is work. And people throw their hands up and say, well, it’s a one-off, so decision frameworks don’t apply. Of course they do. Or they say, it’s a one-off, so I’ll just go with my gut. Please don’t do that. Your gut might be really good, it might be really bad — we’ll never know because you left it in your gut.

Radical Uncertainty

Stephen: And I think one other thing people struggle with around underlying probabilities — and just thinking probabilistically — is this tendency to feel that some decisions fall into a bucket of “radical uncertainty,” where the probabilities can’t be determined in any meaningful sense, and that by thinking probabilistically you’re generating a fantasy rather than anything useful. What are your thoughts on that?

Annie: There is no decision you have ever made that doesn’t involve a forecast. Period. A forecast is some educated guess about what the set of outcomes might be given any choice you’re making, and what the probability associated with those outcomes is — even in cases of radical uncertainty. The reason I know that is: even when you’re in radical uncertainty, if you choose A over B, what’s implied is that you think A has a higher ROI than B. And I don’t mean just money — it could be happiness, fulfilment. This is where we get into expected value, where value means what you’re trying to get out of whatever you’re choosing.

Stephen: There’s always a caricature made of probabilistic thinking.

Annie: Yes, that it’s somehow heartless and doesn’t take love into the picture.

Stephen: Yes, and people zone in on that word “value.” Even if you want to make it as emotionally laden as you like — I always think of Emily Falk’s question, which I steal all the time: would you rather cuddle a puppy for ten minutes or have ten dollars? What I take from it is that we put things on a common value scale already. And when you ask it, people are so quick and sure in having an answer. What’s more interesting than the answer is the fact that they can answer it at all.

Annie: Because we translate these things across all the time. This idea that the uncertainty can be so radical that you shouldn’t think probabilistically — but you are thinking probabilistically. That’s the point. No matter how uncertain the world is, you’re making choices. One choice people make is to not change anything. Implied in that is that the path you’re already on is going to get you to the best place. There’s all sorts of biases that push you toward sticking with your current path independent of whether it’s better for you, but let’s ignore that for a second. Even if you’ve got a choice between two foods you’ve never tried — I guess we’re in radical uncertainty — except that you’re thinking, maybe this tastes like chicken. You’re making educated guesses any time you choose one option over another. Is it going to be right in the sense that two plus two equals four? Of course not. That’s the whole point of uncertainty. The less uncertainty there is, the more you get into two plus two equals four territory. Like if I’m an inch from another car, my forecast of the probability that I’ll hit it if I put my foot on the gas is going to be close to certain. But you’re still making a forecast, because the last time I checked you’re still acting, still deciding, still choosing. And implied in that choice is that you have made a forecast. So I’m saying: make it explicit. Not in the sense that you’ve found the right model or that you’ve fallen for false precision — I don’t want that at all. But make the forecast explicit and make clear why you’re forecasting that. Why are you choosing this? So it can be examined like an object, and other people can examine it and give their input. And it is most important when you’re in radical uncertainty. Because when you accept that there is no forecast to be made and say instead, no — I’m going to make one anyway, so I might as well make it explicit — that’s actually a huge competitive advantage. Everybody’s pretty okay when you’re an inch from a bumper. The more uncertainty there is, the bigger your advantage from getting disciplined about not accepting “I’m just going to wing it.” Part of it is accepting that you don’t know, and so you’re writing down your best guess with your rationale and then getting other people to help you with it.

Stephen: And I think that winging it point is also important — even if someone is feeling unconvinced by all of this in cases of radical uncertainty, what’s the alternative? What’s the better alternative? That’s another way of thinking about it.

Annie: Right. People are so uncertain about the future of AI — what impact will it have, what jobs will it take away, what will happen to electricity bills? It’s a new technology and we don’t really know what the future looks like. Not just in terms of AI’s capabilities or how businesses will respond with their workforce, but also whether governments will regulate it, and how. It’s all reflected in the valuations — there’s a huge fat right tail of returns, but also a lot of these things end up being duds, and we don’t know where we are yet. But if you are going to college and choosing a major right now, among all this uncertainty around AI, I assume you’re choosing the major that will give you the most value, whatever that might be. If you’re thinking about employability, you’re making a guess at what you think the future of AI is — as uncertain as that might be. Make those guesses explicit. Go talk to other people who might have different points of view. Don’t tell them what you think first. Compare those points of view. Explore that. And don’t just throw your hands up and say, well, we don’t know the answer. You don’t know the answer about anything. And I know that because fifteen years ago, everybody became a coder because that was a sure thing.

Closing Thoughts

Stephen: Thank you so much. One final question for you. If someone wanted to take more responsibility for the quality of their decisions and wanted to start doing that, what’s the one small thing you would recommend?

Annie: We’ve talked about a bunch of them, so listen to what we’ve discussed — that will help. But the biggest thing, particularly for anyone making group decisions — and again, everything is a group decision — is: stop telling people what you think when you’re trying to find out what they think. That’s the biggest thing you can do. Within the clients I consult for, we put them into an asynchronous independent model of eliciting feedback. We’re not all yelling in a room what we think. We’re writing it all down first, where I don’t know what Stephen thinks when I’m writing down what I think. But you can do this in really simple ways. If I’m thinking about what major to choose, I could have a set of structured questions I send to Stephen and have him answer them. Or in a conversation, I can say: so what do you think? What do you think the impact of AI is? But notice I didn’t tell you what I think. Surely I have an opinion. But I need to keep it away from you if I really want good information from you. What people do is not that. They’ll say, okay, I’ve been thinking about this... and it’s very hard socially. And the first response they get is often: well, what do you think? Sometimes it’s easier to just send a survey — takes some of the social awkwardness away. But I put everybody into that model because so much of the way we’re biased has to do with contamination from other people’s opinions. And I might suffer from confirmation bias, and you might too — but the things we’re trying to confirm are different. So that’s really helpful. But if I tell you my belief, I’ve now infected you with it. That’s actually going to cause a lot of problems in finding out that you have a very different viewpoint. Everybody should maximise the amount of disagreement they see. And you can’t do that if you’re talking at the same time.

Stephen: I would encourage everyone to do the same. You’ll catch yourself doing it all the time once you start thinking about it more explicitly. As I said, it’s really hard.

Annie: All the time. It’s the natural thing to do. “I love that movie. What did you think of it?” That’s a low-stakes decision, so I don’t really care if you make the error there. But that’s how we talk about everything.

Stephen: You have a brilliant course on Maven. And I believe you have a new book coming out towards the end of this year. Anything else you would like to plug?

Annie: Yeah, the new cohort has opened on Maven — that would be cohort eleven. People can go to Maven and sign up. I think it’s a fun class. I like teaching it. The new book — I’ll be done turning it into the publisher this summer, so I’m guessing Q1 2027. The book is about how to become a better consumer of information — to take responsibility for the way you are interpreting the data you’re coming across. There’s a lot of focus right now on misinformation, but misinformation isn’t actually very prevalent online. Most of what people tell you is true. The problem is that it’s an incomplete picture — not necessarily because they’re malign actors, but because that’s the information they have. So I’m trying to give people the tools and the desire to take responsibility for the way they’re consuming information, and the questions they’re asking of that information before drawing conclusions that will inform their decisions. A simple example: go scroll Instagram, and most of the information you see is before-and-after data. I took a supplement, my cold went away in a week. But what about people who didn’t take the supplement? In the business world: sales were here, I did a marketing campaign, then sales went up — and I’ll show you someone who doesn’t think the campaign caused the increase. It may have. But that data does not tell you that. What if I said sales were here, Ted Lasso premiered, and then sales went up? Nobody falls for it then — but the data tells you just as much in both cases. One is just narratively satisfying. So I’m trying to give people simple tools to stop falling for these statistical illusions. I’m also not a pessimist. Most of the errors are just bonus videos. Most of my course is actually: here are practical things you can do to make your decision making better. The Alliance for Decision Education would also be great for people to check out — it’s a nonprofit I co-founded with my husband. We’re trying to bring decision-making skills into K through 12. Why did you have to wait until adulthood to start learning these skills? We never tell people what to think — the focus is never on that. Two people can look at the exact same information, have great decision processes, and reach a totally different conclusion, because what you value really matters. We would never tell someone what they’re supposed to value or decide. We’re just trying to help them understand how you might go about it.

Stephen: Annie, it’s been a pleasure.

Annie: It’s really nice seeing you again. This has been a super fun conversation. Thank you.

---

If you enjoyed this course you may wish to check out Annie’s excellent decision-making course available on Maven.

If you want to think more carefully about how you make decisions, this newsletter is free. Every Monday morning, something short to help you start the week deciding a little better.

If you are interested in understanding about the decision process more generally, I recommend reading my decision framework.

Share this post with a friend or colleague who wants to make better decisions.