The Peter Attia Drive - Improve your decision-making, frameworks for learning, backcasting, and more ｜ Annie Duke (#60 rebroadcast)

00:00:00.000 Hey, everyone. Welcome to the Drive podcast. I'm your host, Peter Atiyah. This podcast,

00:00:16.540 my website, and my weekly newsletter all focus on the goal of translating the science of longevity

00:00:21.520 into something accessible for everyone. Our goal is to provide the best content in health and

00:00:26.720 wellness, and we've established a great team of analysts to make this happen. It is extremely

00:00:31.660 important to me to provide all of this content without relying on paid ads. To do this, our work

00:00:36.960 is made entirely possible by our members, and in return, we offer exclusive member-only content

00:00:42.700 and benefits above and beyond what is available for free. If you want to take your knowledge of

00:00:47.940 this space to the next level, it's our goal to ensure members get back much more than the price

00:00:53.200 of a subscription. If you want to learn more about the benefits of our premium membership,

00:00:58.020 head over to peteratiyahmd.com forward slash subscribe. Welcome to a special episode of

00:01:06.180 The Drive. For this week's episode, we want to rebroadcast a previous episode that I did with

00:01:10.980 Annie Duke that first aired in July of 2019. I reached out to Annie, a decision strategist and

00:01:17.140 World Series of Poker champion, to have a conversation after reading her bestselling book,

00:01:21.080 Thinking in Bets, Making Smarter Decisions When You Don't Have All the Facts. It was one of those

00:01:26.340 reads that I just couldn't put down once I picked it up, and I wanted to share it with everyone.

00:01:30.540 In this conversation, we go deep into certainty, decision-making, and of course, poker. We talk

00:01:35.340 about why poker is, in many ways, the model system for decision-making in the real world, i.e. in a world

00:01:40.700 where we have incomplete information. We also discuss backcasting, a topic I have since written and

00:01:46.240 spoken at length about, and the importance of doing a pre-mortem, not just a post-mortem.

00:01:51.500 We discuss the decision matrix, meaning how you can look at and evaluate decisions after the fact,

00:01:56.620 when you have to be critical of both the decision-making process and the outcome.

00:02:00.980 I think this episode remains just as important today as it did to me on the day when we recorded

00:02:06.840 it over five years ago. And it contains a lot of valuable and actionable information that you can

00:02:11.800 carry into your day-to-day decision-making and behaviors. So without further delay, please enjoy

00:02:16.320 or re-enjoy my conversation with Annie Duke.

00:02:24.380 Annie, thanks so much for coming all the way up to New York. I'd like to say just to see me,

00:02:28.560 but now you've let the cat out of the bag that I'm the second excuse to be here today.

00:02:32.600 Yes. When I get requests, I try to pack them into the same day for efficiency purposes.

00:02:37.400 Are you from Philly originally?

00:02:38.640 I'm not from Philly, but my father is from Philly originally. So yeah, my dad grew up in

00:02:44.100 Philadelphia and graduated from West Philly High, actually. So he's born and bred. And I live in

00:02:50.680 Philadelphia now. It's actually my third time through. So there's clearly some kind of magnetic

00:02:55.980 center to Philadelphia for me.

00:02:57.920 Are you an Eagles fan?

00:02:58.980 I am.

00:02:59.440 I'm sorry. I can't. I'm so angry about them beating the Patriots two years ago. I'm just so upset about it.

00:03:05.080 Please. Oh, I'm so sorry for you that they didn't get their sixth Super Bowl that day

00:03:10.660 and they had to wait two more years.

00:03:12.940 I don't know. It just bothered me.

00:03:14.260 All right. So wait, are you a Patriots fan?

00:03:16.860 I am.

00:03:17.320 Okay. So interestingly enough, I am a Red Sox fan because I grew up in New England.

00:03:21.040 And it's kind of just an accident of sort of the timing of my own life that I'm an Eagles fan,

00:03:28.120 which has to do with I was never much interested in football when I was growing up,

00:03:31.760 which isn't that crazy for when I was growing up in terms of the Patriots because they were

00:03:36.320 awful.

00:03:37.320 Just horrible. It was horrible. So I like the Celtics because Larry Bird and Kevin McHale and

00:03:43.780 obviously the Red Sox. It was like Fred Lynn and Carl Yastrzemski and Carlton Fisk and like that

00:03:49.400 amazing team. So, and we used to go like once a year. So I would like to remind you as you're so

00:03:55.840 upset that they didn't get number six.

00:03:58.940 On that day.

00:03:59.400 Of what it was like to be a Red Sox fan.

00:04:02.040 I know. I know. Well, you brought up Fisk, right? So it's like, yeah.

00:04:04.860 Was it the 79 team or the 76 team?

00:04:08.040 I think it was 76, but someone's going to correct me, but yeah.

00:04:10.740 Where it's like almost.

00:04:12.160 No, I know. That's, it's hard to watch. You know what it is? I'm a Belichick fan. So people sort of say,

00:04:16.720 oh, you're a Patriots fan. Like wow, wow, wow. But it's not that I disproportionately like anyone

00:04:21.280 on that team. Although I think Tom Brady is, I think unequivocally the greatest quarterback of

00:04:25.900 all time at this point. But to me, it's just, it's Belichick. Like I'm obsessed with the way

00:04:31.660 this guy operates. Like you always get asked the question, like if you could meet anybody famous,

00:04:36.020 who would it be? I mean, at the top of my list would have to be him. Like, and again,

00:04:40.740 it wouldn't be in the setting of him giving glib answers in a news conference. Cause that wouldn't be

00:04:45.060 that much fun. Although it's funny as hell to watch. If I could spend a day on his boat with him,

00:04:50.240 really understanding the discipline that he brings to what he does, it would just,

00:04:54.200 it would just amaze me. There's a beautiful article that was written probably five years ago.

00:04:59.000 It was just told in a really funny way. Cause it was the gist of the article was everybody's got a

00:05:03.140 Belichick story and it's players, coaches, all telling these stories of him that really highlight

00:05:10.500 this single-minded focus of winning that is kind of amazing to me. Even though from a life ethos

00:05:16.940 standpoint, I don't really care that much about sports anymore. I've sort of, there was a day

00:05:20.780 when it really upset me if my team lost. Now it's sort of, Oh, that sucks. 0.96

00:05:24.520 Well, it's good that you've managed to come to that point. 0.98

00:05:28.320 Oh my God. I could tell you stories about when I was in high school, if my team lost, it was like,

00:05:32.760 if you want to get at least one degree, just one degree from Bill Belichick,

00:05:36.400 you should have Mike Lombardi on who wrote Gridiron Genius. It's a great book about leadership.

00:05:42.480 He worked with Bill Walsh, he worked with Belichick. He really kind of talks about

00:05:47.820 what the leadership qualities of those two particular coaches. He's really great. You

00:05:53.060 should look him up. Do you know him? I do.

00:05:54.940 Well, then I'm going to ask you to make that introduction. I will make that introduction.

00:05:58.000 I met Bill Walsh about a month before he died and it was the most sort of simple,

00:06:03.520 he was at Stanford and I guess he was getting some medical care and I was there.

00:06:08.160 I lived in the Bay area at the time, but I still swim at the Stanford pool. And one day I was just

00:06:12.400 finishing a swim and I was walking back to the parking lot and I can't believe it. But Bill Walsh

00:06:17.480 is like walking alone, like on this part of the campus. I normally would never interrupt somebody

00:06:22.300 to be the fan boy because I just feel like it's sort of, I don't know, I just, that's not my style

00:06:26.300 to do it, but I couldn't help it. And I was, Oh my God, I just want to shake your hand. And,

00:06:30.860 and of course, like a total idiot, I just started telling him all of my favorite moments of his 0.99

00:06:35.860 coaching career as though he doesn't remember them. Like, do you remember the drive at the end of the 0.98

00:06:42.240 second Bengals Superbowl when there were only two minutes and four seconds left? And remember when

00:06:47.040 Taylor, and he was just the kindest, he totally entertained all of my nonsense. Like, yeah,

00:06:52.840 those were some great times. Those are some great memories. And sadly he died like a month later.

00:06:57.260 I mean, he was sick in that moment. He was totally able to, anyway. So, but that said,

00:07:01.220 I'm not happy about the Eagles and we'll just let that slide.

00:07:05.220 Literally, it was like two years later. Here's the thing. Belichick is in no danger of not getting

00:07:10.640 into the Hall of Fame. Neither is Tom Brady. This is what I'm trying to remind you of the Red Sox,

00:07:17.620 letting the Eagles have their one moment.

00:07:19.840 Yeah. Well now the Red Sox can't stop doing it. It's also sort of.

00:07:22.560 Yeah. Cause I'd like to remind you that the Eagles did get their first try and like ever

00:07:27.360 at the Superbowl against the Patriots in the first place and lost. So like, come on,

00:07:32.320 it's like crumbs, crumbs for the peasants. 0.93

00:07:36.540 So first of all, I read your book. I don't know how long ago it's been, but I mean, it was

00:07:40.080 probably 10 minutes before I reached out to you on Twitter. I loved it. And I think we all sort of

00:07:44.740 read books and apply our own filter to them. And of course, one of the filters is on this business

00:07:49.960 of decision-making. And in medicine, we make decisions all the time and we make decisions

00:07:55.360 within complete information. And I remember explaining to somebody once something that I

00:07:59.980 think I either heard you talk about on a podcast after the fact and describe it more eloquently.

00:08:03.800 And I think you also alluded to it in the book, which is people always think of chess as this

00:08:08.660 perfect game for decision-making. It's actually my wife. I was having this discussion with. I said,

00:08:12.880 no, actually chess is not a great example because everybody has perfect knowledge at every moment in

00:08:17.840 the game. So it's not to discount the importance of strategy in chess. People who are brilliant at

00:08:22.880 chess are brilliant, but it's not the perfect analog for decision-making in life. For that,

00:08:28.380 poker is much better because you have incomplete information and that tends to mimic what we see

00:08:33.980 in life. And in medicine, I think you almost never have complete information. And that's sort of one

00:08:39.380 of the things that is both frustrating because how can you make a serious decision when you don't know

00:08:44.980 all of the facts. But at the same time, I think makes it intriguing and interesting and gives you

00:08:50.320 this sense of there's a craft that can be mastered here that is less sort of amenable to robotic

00:08:56.140 decision-making. Chess is missing this particular element, which decision-making in general has,

00:09:02.020 which has to do with incomplete information. So in chess, I can see the player's whole position.

00:09:07.460 And what that means is that I should be able to calculate out if I'm considering a move,

00:09:11.680 I know what all the possible moves are that the player can make. And then I can think of what all

00:09:15.860 the possible moves are that I can make in response and so on and so forth, so that I should be able

00:09:19.820 to calculate out with absolute perfect knowledge here what the right move is at any given time.

00:09:25.160 It's missing another element, though, that I think is really important to think about,

00:09:28.940 which is a very strong influence of luck. So in chess, the pieces stay where they are. Nobody takes dice,

00:09:36.940 you roll an 11, and two pawns come off the board, or if it's snake guys, checkmate. Or maybe something

00:09:43.640 good happens, you roll a six, and you get an extra queen. That doesn't happen in chess, so that there's

00:09:49.980 this, we know that the pieces are only ever going to move through an active skill. And this idea of the

00:09:55.580 strong influence of luck is really, really important. Meaning, if I were to do the same thing in the same

00:10:02.120 situation, and then post-decision, are there other things that could reasonably happen that I don't have

00:10:09.300 control over? I don't have control over which thing is actually going to occur. It's actually a really

00:10:13.280 important element to the way we make decisions. If you think about it in poker, if I put my money in with aces,

00:10:20.120 and you put your money in with fives, you know, 80-ish percent to win there is slightly better. That means 80% of

00:10:26.260 the time I'm going to win. But that doesn't mean that you can't win. 20% of the time you can win. And I have

00:10:32.020 no control either way. I don't know if on that particular iteration, I'm going to see the 80%

00:10:37.120 or the 20%. That really crosses all decision-making. So you can tell me this would be my intuition as

00:10:44.180 not a medical practitioner. But not only are you dealing with incomplete information in medicine,

00:10:50.040 but even if you did have complete information, if you apply a treatment in a situation, and it happens

00:10:55.600 to work, there's all sorts of stuff going on, I assume that's relatively stochastic, that we don't

00:11:01.200 really understand, where if we were to do that same thing again, it may not work in the next

00:11:05.720 situation. So we always want to think about that, because that has a very big effect on our decision

00:11:10.440 making. Yeah, you're absolutely correct. And just to hear you explain it that way, that might even be

00:11:15.320 the more important factor, actually, than the absence of complete information. Now, the definition

00:11:20.300 of a chaotic system versus just a stochastic system is even more profound, where even a slight change in

00:11:25.780 the initial condition can produce a profound change in the outcome. We don't even have to

00:11:30.540 posit that we can just leave it at the same initial conditions can produce different outcomes, right?

00:11:36.080 We can kind of unpeel it in this particular way. Let's say that we're dealing in a world where we

00:11:41.100 had perfect information. I think it's important to think about this to understand why chess is really

00:11:45.680 so bad in terms of thinking about these kinds of decisions. It's an interesting decision making

00:11:50.960 problem. I'm not trying to slam chess. I think it's a great game. And it teaches critical thinking

00:11:55.420 skills. And it teaches you to think many moves ahead and to think about what other people know.

00:12:00.020 And there's all sorts of great benefits that it has. It just happens to be missing a couple things

00:12:04.040 that I think are really important. But let's say that we had perfect information. For example,

00:12:08.700 if I have a coin, and I've weighed it, so that I know that the weight is correct, I've examined it,

00:12:17.080 and I can see that it's two-sided, then I now have perfect information. I know exactly what I need

00:12:24.100 to know about the coin to understand how often it's going to flip heads and how often it's going to

00:12:29.480 land on tails. But what I can't know is what it will land on the next try. So that's that issue of

00:12:35.480 luck. So if you flip and it lands heads, it's reasonable for me to say it could have landed tails.

00:12:41.560 And if we did that again, that tails is totally a possibility, regardless of what I happen to have

00:12:47.160 seen on this particular flip. So and then what we can do is say, okay, so even in conditions of

00:12:53.840 perfect information, we don't really know which outcome we're going to get, we might know the

00:12:57.460 frequency, how often that might occur, how often we could expect it to occur, but we don't know for

00:13:02.260 sure what will happen this time. And then you can take it further and say, but what if you haven't

00:13:07.020 examined the coin? And based on some set of outcomes, you're now trying to derive what the

00:13:12.720 coin looks like. Now that becomes even harder. And that's where you get into something that looks

00:13:18.800 very little like chess.

00:13:20.520 Why is it do you think that as humans, we, at least most people, most of us don't seem innately wired

00:13:28.760 to understand probability theory? Because I studied math as an undergrad, and I spent so much time in it,

00:13:34.520 I think I take for granted now how easy it is for me to think probabilistically. I sort of view the

00:13:40.940 world as a stochastic map, but I'm pretty sure I wasn't born that way. And I'm pretty sure that 0.87

00:13:45.900 nothing, maybe this isn't correct, but I doubt that we were evolutionarily wired to do that. Or I feel

00:13:51.540 like we would see it more often in the way people behave. And I feel like the absence of that in the

00:13:56.440 general person's point of view, who hasn't been forced to be trained in that way, speaks to that.

00:14:01.820 This strikes me as one of the more difficult things in explaining anything. This touches

00:14:06.140 everything from politics to business, to science, because it's too easy to say,

00:14:13.640 well, that didn't happen, therefore, that couldn't have happened, as opposed to describing it the way

00:14:19.360 you did, which is no, actually, it could have happened. And if the same initial circumstance

00:14:22.700 happens again, by the way, this thing could happen.

00:14:24.260 I would imagine if we were to take the distribution of how naturally probabilistic

00:14:31.120 thinking comes to somebody, I'm guessing that you were probably born closer to the right tail.

00:14:36.480 That being said, I guarantee you that if I followed you around for a day, I could find all sorts of

00:14:41.360 spots where you're not thinking probabilistically. And I could in particular find all sorts of spots

00:14:46.760 where your initial gut reaction is not probabilistic and you have to stop yourself.

00:14:52.580 I'm positive. I'll give you an example of a game I played that's not an exact example of this,

00:14:58.060 but it humbled the hell out of me. And I always go back and play this with other people.

00:15:03.340 So the idea of the game was to teach people what a 95% confidence interval looks like.

00:15:07.960 And this is back when I was at a consulting firm called McKinsey and Company, and I was a part of

00:15:11.440 the risk practice. Our whole job was walking around and helping companies manage risk.

00:15:15.700 So we did this at an offsite one day to help us kind of figure this out. So the moderator goes up

00:15:22.060 with 20 questions. Have you played this game?

00:15:24.340 So I have, and this is one of Phil Tetlock's favorite who wrote Expert Political Judgment

00:15:29.380 and Super Forecasting, which I highly recommend anybody interested in probabilistic thinking.

00:15:33.960 I ended up reading Super Forecasting years after this experiment.

00:15:37.180 Yeah. So you know, he's got this in there. Yeah.

00:15:38.760 So I'll tell it just for the listener. So you take, and I still have it, by the way,

00:15:41.480 I still have the 20 questions.

00:15:42.720 I was terrible at this.

00:15:43.760 I'll tell you what my score was after I'm so pissed about it too, like how bad I was.

00:15:47.260 Mine was like zero. So go ahead. And I've really tried. It's just like the questions that I got

00:15:52.860 were really out of my knowledge base. And it was like, I was trying to have really wide,

00:15:56.460 but it was a disaster.

00:15:58.080 Okay. So what the moderator does is they give you 20 questions. Each one has a quantitative

00:16:02.840 answer that is known.

00:16:04.860 That is known. Yeah.

00:16:05.640 And the question for you, the participant is for each question, give me a bracket that contains

00:16:12.440 the answer. So for example, what is the average height of a male in the United States? An

00:16:19.680 appropriate answer would be somewhere between four foot six and six foot one, because that would

00:16:26.580 be a reasonable guess at a bracket that contains the average height of a male.

00:16:31.820 Right. So the way that I've heard it is that you want the bracket to contain it 95% of the time.

00:16:36.480 Which means after you've done all 20, you should have exactly one incorrect and 19 correct. And

00:16:44.600 the purpose of saying 95% confidence is if they said to you, you have to have a hundred percent

00:16:50.120 confidence. You can sandbag it one inch tall. Yeah. There's somewhere from one inch tall to a

00:16:54.560 million miles tall. And the questions in fairness to people like you and I who have flunked the test,

00:16:59.160 the questions are hard.

00:17:00.440 So did you get, how many votes did Abraham Lincoln receive?

00:17:03.500 I didn't get that one. I got like, what is the distance between Jupiter and its nearest

00:17:07.900 moon? It's like, I don't have any goddamn clue, but 0.84

00:17:10.820 how many people were voting during Lincoln's? I don't know. And I had this really wide range 0.97

00:17:17.500 and that was great.

00:17:18.260 But it's such a great exercise because you think, okay, I've got this under control. I'm going to

00:17:24.200 just try my best to get everyone correct with a reasonable guess. And I'll probably get one wrong.

00:17:30.480 And I think in the end I got 12 right and eight wrong, which of course is like a 60% confidence

00:17:36.660 interval on things that are perfectly knowable. And so, yes, I completely agree with you. And of

00:17:42.220 course your initial example is a better one, which is how many times do I make either a decision or

00:17:48.840 have a gut reaction that I have to actively talk myself out of?

00:17:53.400 The reason why I bring that up is that I think that there's a couple of things that we want to sort

00:17:57.260 of keep in mind. And then if you want to talk about sort of like from a, like an evolutionary story,

00:18:02.000 why it is that we're not particularly wired to think probabilistically, we can get into that.

00:18:06.980 But what I want to get across is this, that first of all, we're all sort of like sitting,

00:18:11.660 we have proclivities, you know, where we're sitting on a distribution, somewhere on the distribution in

00:18:16.640 terms of sort of how we're born, how we're wired to think probabilistically. And that doesn't mean

00:18:23.480 that we should say either we're there or we're not. I mean, the idea is that wherever we sit on

00:18:28.980 the distribution or whatever our personal distribution looks like, we just want to shift

00:18:33.420 it to the right. So wherever you are, you want to accomplish two things. One is it would be really

00:18:40.040 good if you thought probabilistically a little bit more. Why? Because the world is probabilistic.

00:18:47.680 And so the more accurately you can think about what the outcomes of your decisions might be,

00:18:53.520 because they're not deterministic. When you make a decision, there's many, many things that could

00:18:58.480 occur with some likelihood of each of those things occurring. And understanding that particular fact

00:19:02.840 is incredibly important to good decision making. So the more that you can shift yourself a little bit

00:19:07.060 more on the side of probabilistic thinking, the better off you are, no matter where you've started.

00:19:11.020 And then the second thing we want to accomplish, which is what I was trying to get at,

00:19:13.820 at the catching yourself part, is to understand that we are wired this way. So it's really hard

00:19:20.740 at all times to think that you don't know things for certain, to sort of live in this counterfactual

00:19:28.260 world. And it's frankly kind of impractical. It would be hard to like drive, for example, if we were

00:19:33.260 living counterfactually all the time. So the idea is that when we do make an error, to catch it faster.

00:19:39.540 Because for most things, when we make an error, and we have too much confidence, or we're thinking

00:19:45.940 about something as deterministic, when we should be thinking about it as probabilistic, for most

00:19:50.920 things, we don't catch it at all. So if we can just catch it a little bit more often, and if we can

00:19:56.860 catch it a little bit more quickly, we're a lot better off. And these small shifts, if you can get a

00:20:03.340 couple percentage points better, you're way better off in your decision making.

00:20:08.500 So let's go back to kind of where you started thinking about this, which was through the lens

00:20:12.600 of playing competitive poker. How did you end up playing poker?

00:20:16.380 Chaos theory. So let me just say that when I started playing poker, it wasn't all over television,

00:20:21.340 because I think that otherwise, that would be a really simple answer. Like I was in college,

00:20:24.940 and I wanted to make money. And I saw poker on TV, and I started playing online,

00:20:29.660 I think would be the origin story of the majority of poker players today. Not my origin story,

00:20:35.560 because I'm way too old for that.

00:20:37.100 Where did you go to college?

00:20:37.920 Columbia.

00:20:38.400 Okay.

00:20:38.780 Yeah.

00:20:39.120 So you're here in the city.

00:20:40.140 So I'm here in the city, and there's no internet. I mean, I think there's some sort of, I don't know,

00:20:45.520 Al Gore was inventing it, or something. I don't know, but it didn't exist. So the first thing that

00:20:50.540 happened on my way to becoming a professional poker player is that my brother, when he was in high

00:20:56.740 school, got very, very interested in chess, started really studying it, became incredibly good at the

00:21:02.360 game. He got accepted to Columbia, decided to take a year deferment to live in New York and study

00:21:10.300 with a grandmaster. During that year, he started playing poker. All sorts of things happened. He

00:21:16.380 lost his college nest egg. He eventually did very well. But in this particular year, he was learning,

00:21:23.560 let's say, and he was learning maybe on the money that had been saved for college. It was a little

00:21:28.440 bit of money. It wasn't a lot. Anyway, he had started playing, and then I followed to New York

00:21:33.440 to go to Columbia a year later. I actually went. I didn't defer. And by the end of that year, my

00:21:38.840 brother was very good. So now we're both living in New York at the same time. He actually doesn't

00:21:43.740 end up going. Much later, he ended up going, I think, for like six months, but he ends up never

00:21:48.060 finishing college. And he's playing poker professionally. And I'm going and watching

00:21:53.000 him sometimes. So I'm just sort of sitting behind him.

00:21:55.140 And what does that mean at this era of time? These are like smoky rooms in New York City?

00:21:59.500 Yeah. There was a place called the Mayfair, which had a game. And they had poker, and they

00:22:06.500 had backgammon. Before that, my brother was playing at a place called the Bar Point. For

00:22:12.060 people who don't know, that's a backgammon term. And it was like a backgammon club. People

00:22:17.240 also played gin there. It was on 14th and 6th. And in the back room, there was a poker

00:22:23.520 game. This was before I came around. He was sort of already out of this place by the time

00:22:27.440 that I came around. Although I did spend a summer here where I got to see him sort of

00:22:31.380 playing in that particular place. So he found out about that through like his interest in

00:22:36.340 chess, ends up there in the back room. There's a game that starts on Friday night and ends

00:22:40.960 on Monday morning. And by that, I mean, that's how long the people played. They didn't get

00:22:46.780 up and leave and come back.

00:22:48.140 We can have a whole other discussion about sleep deprivation and decision making, but

00:22:51.740 we'll save that for another time.

00:22:52.820 Yeah. This was in the early 80s. So I believe there was cocaine involved.

00:22:56.880 Yeah, I'm sure.

00:22:57.800 But at any rate, so he starts playing in that game. And that's actually where he...

00:23:01.620 So he cuts his teeth. That's his apprenticeship.

00:23:03.120 That's where he's cutting his teeth. That's where he's sort of learning how to play. That's

00:23:06.260 where he's losing this college money. And eventually, so he's playing in that game.

00:23:10.840 He's not doing very well. And he starts figuring out, oh, actually, it's sort of in the same

00:23:16.160 way he approached chess. He sort of found whatever books he could find. And he found

00:23:20.520 a few other people who were kind of really interested in learning the game. And he sort

00:23:24.040 of starts working the game. And eventually, he basically sort of breaks that game. Like

00:23:30.540 he becomes so good that I can't remember whether they kicked him out or not. But in the meantime,

00:23:35.500 when he had lost his college money and what he starts doing, because now he needs money

00:23:39.440 to be able to play, is he became like the errand person for the game. And so they'd be

00:23:45.980 like, hey, Howard, will you go get me a sandwich like at the deli downstairs or go get me some

00:23:51.060 pizza or go get me cigarettes or go whatever. And he'd go and do an errand and then he'd

00:23:55.280 get a tip. And then he'd save his tips up and then get into the game. Eventually, he didn't

00:23:59.680 need to be the errand person anymore because he turned the tip money into enough money for

00:24:04.240 him to eventually send himself to college for that brief period. But so he moved from

00:24:08.640 there to the Mayfair, which was...

00:24:10.780 Is it in Gramercy Park? Is it near there? It's either Gramercy or Chelsea. I think it's

00:24:16.760 Gramercy Park. I think it's near Gramercy Park. It's a nice place. Then he was playing

00:24:20.560 there for a while. And he falls in at that point with a group that really becomes very

00:24:25.680 famous. Dan Harrington, main event champion. Eric Seidel, who I think has nine world championship

00:24:33.000 bracelets from the World Series of Poker. It might be more than that now. But he's won,

00:24:36.540 I think, like $40 million playing poker. I mean, he's amazing. Steve Zolotow, also bracelet holder,

00:24:42.660 was in that group. Jason Lester.

00:24:44.460 And these guys are all coming out of this one group. So this is sort of like Daniel Coyle

00:24:48.940 writes about these centers of excellence, or I forget what his term is for them, but these

00:24:52.740 little places that just produce so much greatness, like all these soccer players that come out

00:24:57.720 of this place in Brazil and all these tennis players that come out of this place in Russia.

00:25:02.020 This became the place where everybody's sharpening everybody in this one

00:25:06.520 place. If you're a tourist and you hear about this game, you don't get to go and play,

00:25:10.360 obviously, right? Like they'll take your money if you want, but...

00:25:12.680 Yeah, anybody can play. And anybody does play. The thing about poker that's part of this stuff

00:25:18.280 about like the hidden information and luck is that that kind of uncertainty leaves open the

00:25:24.000 array of explanations for why you might not be doing well. So if I play chess against you a few times,

00:25:30.600 it becomes very clear to me why I'm losing. I'm losing because you're better than I am.

00:25:34.580 I have nothing else to protect my ego. Like I can't put a hat on it and say,

00:25:38.920 no, it's because of something else.

00:25:40.840 See, that's such a great point. That's worth pausing on it as well for a moment.

00:25:44.460 There are very few sports or activities that we play that are as pure as something like chess,

00:25:52.620 where as you said, there is zero excuse. Whereas I can't think of another activity. I'm sure there

00:26:00.420 are several where if you really want to, you can't, oh, the ref screwed me or the wind. I mean,

00:26:06.240 did you see that? Or like, I'm into archery, right? It's like, oh, did you see that animal move out of

00:26:10.240 the way at the last second? He heard the arrow coming or whereas in reality, no, you took a lousy

00:26:14.700 shot.

00:26:15.300 Right. I mean, they're all on a spectrum. And one thing that we forget is that there's an equanimity

00:26:19.940 to luck. So if you're playing tennis and there's wind, your opponent is also playing with that wind.

00:26:25.280 And if you're in a game, although we like to think so, the refs generally are making bad calls, 1.00

00:26:30.860 just sort of period, some good, some bad. And they're not like screwing a particular team,

00:26:36.940 although systematically going against a particular team, although it sort of feels like that.

00:26:42.000 But what we notice is the wind blew our ball. An interesting thing that happens in sports that

00:26:47.020 are one-on-one, like I play tennis, is that you can blame it on the style matchup. That it's not that

00:26:51.660 like you're not particularly good.

00:26:52.900 That's a great new excuse. I hadn't even really thought of it.

00:26:54.680 No, you should use that. Like, oh, that was a bad style for me. I mean, I'm actually

00:26:57.660 better than them, but they're lobbing. They were doing a lot of lobbing and

00:27:01.520 my high balls weren't really on that day, but I would crush anybody else. So yeah, exactly. 0.88

00:27:07.160 So poker basically is just a fertile ground for that thinking.

00:27:11.300 Right. Which means that it doesn't cause you to kind of self-select out. As you said,

00:27:16.300 this was like a nest of what was going to become the best players in the world. And of course,

00:27:20.420 people were still coming back. So he ends up at the Mayfair.

00:27:23.760 Well, I guess there's another element here, which is the variable reinforcement of winning.

00:27:28.880 Wow.

00:27:29.280 So then you've got the whole dopamine side of this coupled with this psychology of,

00:27:34.640 I can post hoc rationalize my way all day long. I can rationalize my losses, pump up my winnings

00:27:41.620 from an ego standpoint. And the next win is coming at a point I can't predict. I mean,

00:27:46.500 that is a recipe for getting me into this.

00:27:50.080 Actually, let me just bite into that for a second. So this is actually really interesting. So let me

00:27:54.560 just say, I didn't play poker at all in college. Like I never played. It wasn't until graduate

00:27:59.380 school that I played three times or something. So I go to graduate school at Penn. I'm studying

00:28:03.480 cognitive science. This becomes important to me later as I'm thinking about the game of poker,

00:28:08.500 this issue that you just brought up, the difference between a variable reinforcement schedule

00:28:12.460 and a fixed reinforcement schedule. What we can think about is we can talk about ratio or interval

00:28:19.160 schedule. So ratio schedule would be by number. So if you put a rat in a Skinner box, and for those

00:28:26.740 who don't know, Skinner box, it's got the lever. And then either there's positive reinforcement,

00:28:30.820 like a food pellet comes out, or maybe negative reinforcement, like a shock. So let's make it positive

00:28:35.500 and say it's a pellet. A ratio schedule is by number. So it would be like every fifth press,

00:28:42.000 the rat gets a pellet. And an interval schedule is by time. Every five minutes, the rat gets a pellet.

00:28:49.840 And if it's fixed, what it means is it's actually every five presses or every five minutes. And

00:28:56.120 depending on whether it's ratio or interval, you get different behaviors from the rat. So what happens

00:29:02.420 on the ratio schedule is that the rat has this very even pressing that goes on this very even,

00:29:10.540 somewhat quick, but not mad, press, press, press, press, food, press, press, press, press. And then

00:29:17.660 if it's every five minutes, the rat twiddles its thumbs until it's almost five minutes, and then it goes

00:29:24.260 crazy. So you get this burst of activity. It's almost like they don't want to miss a second.

00:29:29.620 They want to make sure that they get that food exactly. They don't want to give up one second

00:29:34.720 where they could have had food. And so you get this big burst of activity. Then you can also take a

00:29:39.700 ratio or interval schedule. And instead of it being fixed, you can make it variable. And what that means

00:29:44.200 is it's now on average is the way that you can think about that. So let's say on average, every 20

00:29:49.040 lever presses, you'll get food. Or on average, every five minutes, you'll get food. So now instead of this

00:29:58.020 big burst of activity on the interval schedule, on average, every five minutes, because they're not

00:30:03.060 sort of waiting, and then, oh, I think it's about five minutes, and they go. Now they're pressing

00:30:07.960 pretty steadily the whole time, because they're not sure anymore, right? Is it going to be the second

00:30:12.640 or not? And then on the variable ratio schedule, you get the same thing. You get a lot of pressing

00:30:18.300 very quickly, because they don't know, is it the next press or not? So that's all kind of interesting.

00:30:22.480 And you'll see a different activity, depending on whether it's variable or not. What's interesting is

00:30:27.120 when you try to extinguish the behavior. This is what's really key. So when you extinguish a

00:30:32.420 behavior, what you do is you just withdraw the reward. So now you've trained the rat,

00:30:37.940 it's pressing the lever, and then you just stop giving the rat food. And the question,

00:30:41.920 what does it do? When it's fixed, it stops doing the thing super fast. It's like, hey,

00:30:48.060 wait a minute, like I've been pressing 100 presses of this stupid lever, you haven't given me any food, 0.99

00:30:54.160 I give up. I have figured this out. There's no food coming my way. Same thing if you're doing it 0.99

00:31:00.060 by time. But when you make it variable, they just don't stop. And it's particularly bad when it's a

00:31:06.060 variable ratio schedule. So I think about it like this. So anybody who's ever watched people sitting

00:31:13.300 at a bank of slot machines in Las Vegas, those are on a variable ratio schedule, that you'll get a

00:31:18.860 reward, it's going to be on average, a certain number of plays, you could get a reward five plays

00:31:25.040 in a row, you could go 80 plays without really getting much in return. And you kind of don't know

00:31:30.960 because it's on average. And what happens to people sitting at those slot machines is that

00:31:36.140 they'll be going along and there'll be no reward. And you'll hear them say a thing. I'm due.

00:31:42.120 I'm due. And what that means is that it doesn't matter what you do to that machine, you could just

00:31:49.660 turn it off. What they think is no, it must be the next press, it must be the next press. And this is

00:31:56.000 exactly what happens to the rats in the Skinner box, you train them that on average, every 20 presses,

00:32:02.180 they get food. And that means sometimes they're getting three pellets in a row. And sometimes they're

00:32:06.900 going 80 presses without getting food. That's obviously out at the tail, no pun intended, but

00:32:12.540 it's just an average. And now you would draw it and they just never stop pressing the lever.

00:32:18.040 And it looks just like what humans do. And I sort of always imagine the little rats in the cages saying

00:32:22.860 I'm due. So this is exactly the way that something like poker works. If we're thinking about sort of an

00:32:29.540 edge, what that means is that over time, I'm going to get a certain percentage of your money,

00:32:36.440 money. But in a particular moment, you could win a lot of hands in a row. In a particular moment,

00:32:43.460 you could not win so many hands in a row. And unless you're a really good statistical aggregator,

00:32:48.920 right, unless you're sort of stepping back and really looking at what does this look like over

00:32:53.680 the long run, it's really hard to see what the underlying distribution is. It's hard to see whether

00:32:58.100 you're winning or losing to that situation. Well, there's a slight difference, right? Because if I

00:33:01.720 understand correctly, the slot machine is driven by an underlying probability distribution curve.

00:33:08.360 So the person sitting there playing it like the Skinner rat is kind of right in that they are due,

00:33:15.400 they just don't know when. Whereas the person playing poker, there is no certainty that they

00:33:20.180 will ever win again necessarily. In other words, it's both luck and some skill. So they could take

00:33:27.920 probabilistic good hands and misplay them and lose on a potential hand that could have been their due,

00:33:34.640 right? Yes. I'm thinking about that. Yes. Yeah. So it's sort of like in some ways, the person at the

00:33:39.700 poker table has an even harder problem than the person at the slot machine. Yes. Because the slot

00:33:46.200 machine is pure luck. So the slot machine simplifies the problem because the machine is set to a

00:33:53.200 particular distribution. And the probability is set about how much you're losing per dollar. And

00:34:01.160 so if I sit down, the underlying probability is knowable. That's right. And your only task is to

00:34:08.940 pull the lever or push the button. Whereas in poker, you still actually have to do what we're going to get

00:34:15.040 to, which is the skill of this game. Right. And what you're trying to figure out is, I mean,

00:34:20.240 obviously this isn't true of everybody, but when people sit down at a slot machine,

00:34:24.280 people aren't thinking I won because I'm so skillful. I mean, occasionally they are if they

00:34:27.840 think they have a system. I don't think that's true. Most people are saying like, I'm doing this

00:34:31.680 because I'm going to get lucky and maybe I'm due and whatnot. So I think that you have less illusion

00:34:36.380 about it. So now in poker, you lay on top of the fact that it's stochastic, that obviously it's a

00:34:43.740 skill game. And in the long run, with enough iterations, your results will be solely determined

00:34:50.320 by skill. Right. Because the probability starts to approach equal distribution for everybody,

00:34:55.360 assuming there's no cheating. Well, it would be true even if there's cheating for this reason,

00:34:59.520 that what that would mean is that whatever skill elements that you were applying, you wouldn't be

00:35:04.000 able to overcome the situation. So all I mean by this is that you have an expected value.

00:35:09.240 That expected value is determined by your play compared to your opponent's play.

00:35:14.860 And when you say skill, do you mean two skills? The first being the ability to look at all the

00:35:23.300 cards that are available and calculate the probability of cards that are not being shown and the skill of

00:35:31.740 being able to bluff? Or are you referring more to the latter than the former?

00:35:35.500 Skill would be the umbrella of all the things that are within your control in the game that have an

00:35:41.760 effect on the outcome. So first of all, let me just step back a second. Sometimes people think that a

00:35:48.040 game isn't skill if in the short run, there's a really strong influence of luck. So people will say,

00:35:55.460 well, baseball is a game of skill, but poker isn't a game of skill. Because they're looking at,

00:36:01.020 well, but in poker, I could have the best hand, I could play it perfectly, and I can lose to you

00:36:08.320 because of the turn of a card. So therefore, isn't it just luck? And how can you tell if something is

00:36:14.620 a game of skill? Because I'm obviously making the assertion that in the long run, your results are

00:36:19.080 going to be solely determined by skill.

00:36:21.020 I think the way you said it actually is probably the best definition I've ever heard, which is

00:36:23.960 in the long run, if two different people can play the exact same game over and over and over again,

00:36:29.320 and have dramatically different results under the same stochastic input, doesn't that imply that

00:36:36.060 there is a difference that is in the control of the player? And isn't that the definition of skill?

00:36:41.240 Yes. Because of actions that they take, their distribution is different.

00:36:44.860 Right. So for example, if you and I made up a new game, which would be a really boring game

00:36:48.080 called Toss Two Dice, and the person who gets the higher number wins. If you and I played this game

00:36:55.000 for the next week, we sat here at my table, and we played over and over and over again,

00:37:00.080 in the end, we would end up the same.

00:37:01.580 There would be nothing that we could do that would change the outcome.

00:37:03.620 And therefore, there is no skill in that game.

00:37:06.060 There is no skill in that game. But if we can bet on it.

00:37:07.920 That's right. And baseball, of course, is such an obvious example of skill,

00:37:11.400 because while there is luck involved, within a matter of minutes, you would tell the difference

00:37:16.500 between me and a professional baseball player doing anything on the visual field.

00:37:20.200 There's a really great test that you can do to figure out if something is skill. And then let

00:37:25.720 me just get back to baseball for a second to show what's happening, why people are sort of

00:37:30.320 misperceiving with poker, whether it's a skill game or a luck game. So the first thing is you just

00:37:35.320 want to apply a test. Can you lose on purpose? Assuming you followed the rules of the game.

00:37:39.160 Now, it's not symmetric. I cannot apply a test of can I win on purpose, but I can apply a test of can

00:37:45.540 I lose on purpose. And I'll tell you why. So tic-tac-toe, let's agree that that's a skill game.

00:37:50.200 It's a bottom-feeing skill game, but it is.

00:37:52.980 Right. But if you and I play, neither of us can win on purpose, assuming that we both know how

00:37:58.500 to play the game, because we're going to tie every single time. But one of us can lose on purpose.

00:38:02.100 Very easy to lose on purpose at that game. So we don't want to apply the win on purpose,

00:38:06.320 because that's hard, particularly as you narrow the skill gap, then you can't do that. But I can

00:38:09.880 lose on purpose. And in a game like baseball, you can lose on purpose. In a game like basketball,

00:38:16.200 in a game like chess, in any of these games, you can lose on purpose. We can think of other

00:38:21.580 acts of skill. Like if you're a salesperson, you could make it so that on purpose, you never

00:38:26.420 closed a sale. As an example, it's a game of skill. If you were a lawyer, you could probably

00:38:31.280 make sure you lost every case. I mean, hopefully you don't do this, but, and in poker, you can

00:38:36.180 indeed. I can lose on purpose very easily. I can't win on purpose, but I can lose on purpose. And

00:38:41.360 that's true of baseball as well. Somebody can lose on purpose, but they cannot win on purpose

00:38:45.900 that you do not control. That at a basic level is a really good test of whether something is

00:38:52.160 a skill game or something is a luck game. Where I think the confusion comes in is that

00:38:56.360 as you narrow the skill gap, luck starts to play a much bigger role. It starts to kind

00:39:02.760 of show itself that influence of, hey, there's different ways that this could turn out. And

00:39:09.240 sometimes you're going to see one thing and sometimes you're going to see another thing

00:39:12.500 and you kind of don't have control over which thing you happen to see a lot of the time.

00:39:16.360 We start to see that influence as we narrow the skill gap. So that's a great point, right? That's

00:39:21.180 why you look at virtually every Superbowl these days. Most of them come down to the last series,

00:39:27.580 the last possession. I don't remember these stats, but it could be as high as like two thirds

00:39:31.900 of Superbowls come down to the last possession. And if you think about that, you could easily

00:39:37.360 do what I do when looking at the Eagles beating the Patriots and say it came down to that one 1.00

00:39:42.600 stupid play when Brady went back for the pass and boom, the ball gets tripped out of his hand. 0.99

00:39:47.080 And it's like, if that didn't happen, he absolutely would have connected with Gronk. They absolutely 1.00

00:39:51.100 would have won. And whether or not I'm being delusional or not, you can't deny the point that

00:39:55.960 those teams were so damn near perfect.

00:39:58.860 If you took the Red Sox and you had them play a Little League team, there'd be basically no influence

00:40:05.160 of luck, right? Like what luck is going to go the way of the Little League team? I mean,

00:40:10.540 with the Red Sox, what luck is going to go against them? Aside from all their players all at once

00:40:16.880 get injured and can't play anymore, they could put in their third stringers and the Little League team

00:40:22.820 is crushed. But if you put the Red Sox against the Yankees, this is why we have to have seven games.

00:40:32.320 And it's probably still not enough to try to figure out who's supposed to advance to the next

00:40:37.500 round. And in fact, we know it's very rare in a series that a team wins for nothing. It's very

00:40:44.040 often going to the sixth or seventh game. Why? Because it's very narrow. And it's very often

00:40:48.960 things like you just said, who wins the coin toss in a football game that has a huge effect on what

00:40:54.460 the outcome is, who just gets to go first? Especially in overtime. Yeah. Oh my gosh,

00:40:58.760 that's a huge influence of luck. Why? Because you've narrowed the skill gap so much. But if you

00:41:03.500 took that NFL football team, and you put them against peewee players, or even a college team,

00:41:09.180 or even a college team, it's unbelievable. The coin toss would have no effect. No one would care

00:41:13.880 who won the coin toss in overtime. The NFL team, I mean, I don't know how they would get to overtime.

00:41:19.900 But you just magically make overtime appear, you put them in there, and the coin toss will have no

00:41:25.740 effect whatsoever. So I think this is something for people to really understand. And so if we think

00:41:30.400 about a game like poker, and a lot of these decision making processes that we do in life,

00:41:34.560 the skill gap is actually quite narrow. So once I've taught you in poker, things like you know,

00:41:40.240 the ranking of the hands, like a flush beats a straight at you, you know, you can bluff, you know,

00:41:45.900 sort of about betting, you can read your hand, all these things. That's so much of the building

00:41:51.080 blocks of the game in terms of the skill. And how long does it take? Let's hit pause for one

00:41:54.840 second. And just for the person listening to this, who maybe doesn't know the ins and outs

00:41:59.800 of poker, when we're talking about poker, you're talking about a variant of poker called Texas

00:42:03.920 Hold'em, correct? Not necessarily, but we can. Okay, that's what you played, correct? And that's

00:42:08.280 what most of the pros are playing when we turn TV on and stuff like that? So what you see most of the

00:42:12.120 pros playing when you turn the television on is Texas Hold'em. In terms of what most of them are

00:42:16.800 playing, it's not necessarily that game. That's one of the games that they play. So give us the brief

00:42:21.820 overview of how that game works so that we can understand what those building blocks are that

00:42:25.780 you're talking about, which is everybody within a day could know in detail what you're explaining

00:42:31.060 quickly about how this game works. So this isn't true of all poker games, but it's true of almost

00:42:37.060 all poker games enough that we can pretend that it's kind of true across the board that the goal

00:42:41.900 is to make a five card hand. Again, there are some exceptions to this. I just want to make that clear,

00:42:46.280 but I'm capturing most of poker. If I say you want to make a five card hand,

00:42:49.660 those five card hands range from none of the five cards match up. And so all you care about is the

00:42:57.160 top card and the hand. In most poker games, aces are high. In some poker games, they're low,

00:43:05.200 but mostly they're high. Sometimes they're high and low. It depends. So you're trying to make a

00:43:10.100 five card hand. Those five card hands range from very bad, meaning none of the cards match up to each

00:43:16.040 other in any way, to really, really good. And really, really good would be a straight, which

00:43:22.360 would be five cards in rank order. Five, six, seven, eight, nine. That would be a straight.

00:43:29.080 Ten, jack, queen, king, ace. That would be a straight. You can have a flush. That's when all

00:43:33.900 five of the cards are of the same suit. So you have five hearts or five spades or whatever. Full

00:43:38.820 houses where you have three of one rank, king, king, king, and then two of another rank,

00:43:43.800 10, 10. So your five cards would be king, king, king, 10, 10. So anyway, people can go and look

00:43:49.860 up the hand rankings. The highest rank is a straight flush. It's a royal straight flush. So that would

00:43:54.280 be an ace, king, queen, jack, 10, all of the same suit. Probability of that hand. I'm not sure what

00:44:01.420 it is, but a straight flush is one in 6,000 something. It hardly ever happens. So let's start

00:44:08.200 there. So basically almost any poker game you play, you're trying to make a five card hand. And there's a

00:44:12.740 very clear set of rules about what the hierarchy and ranking are. There is no ambiguity. There's

00:44:18.320 no ambiguity when you're playing. Now, different games have different rankings because you can play

00:44:22.180 low. So in low, you're actually trying to make sort of what in high would be a bad hand. Okay. So then

00:44:27.400 the other thing about most poker games is for most poker games, you're trying to make a five card hand

00:44:33.180 out of either seven or nine possible cards. That just is also true of most poker games, not all.

00:44:39.920 So in Texas Hold'em, it's a particular form of poker where you have seven cards to work with

00:44:46.200 to try to make your best five. So you're sort of like tossing two of them. But some of the cards

00:44:54.080 everybody gets to use. They're called community cards. So we can think about the variance of poker.

00:45:00.200 So when you watch something like the Cincinnati kid, for example, Steve McQueen, in that particular game,

00:45:08.580 people are dealt their own five cards. And so you don't get to use any of the five cards that I have.

00:45:14.360 So our cards are unique from each other, and you don't get to see any of my cards.

00:45:19.760 Then you can have variance, which are called stud games, where all of our cards are private,

00:45:25.440 but you get to see some of my cards. So in a stud game, at the end of the hand, I'll have three cards

00:45:32.620 that are face down and four cards that are face up to you. And I'm still making my best five of

00:45:39.000 those seven. And I don't get to use yours. It just gives me more information. It just gives you more

00:45:43.500 information. So once you can see the four cards that are up, that gives you some definition to

00:45:48.360 the kinds of hands that I can have. So as an example, because I only have three down, if my four

00:45:53.840 up cards are a spade, a club, a heart, and a diamond, I can't possibly have a flush, because

00:46:02.040 the most I could have of any suit now is only four. So you get certain information about the

00:46:07.240 possibilities for what I could be holding, because you get to see some of my cards. But yet, you still

00:46:12.040 don't get to use any of my cards. Whereas in Texas Hold'em... Now that's a stud game. But it's common.

00:46:17.300 But it's common. So what happens is you get dealt two cards that are private to you. So these are

00:46:21.920 face down. You don't get to see those particular cards. And then, in a particular way, five cards

00:46:28.760 eventually will end up in the middle. So notice that that's seven. But those five cards that are

00:46:33.380 sitting in the middle, both of us get to use. Now, what that means is that these five cards in the

00:46:39.460 middle define for me what your possible holdings are. So that now changes for me what I think that the

00:46:47.800 strength of my hand is. I'll give you a simple example. It just turns out, if you have two

00:46:52.560 private cards, the most of a suit that I could have in my hand would be two. So I could have like

00:46:57.700 two hearts. Which means that in order for me to ever make a flush, which is five of the same suit,

00:47:03.460 like five hearts, I need to have three more of those out on the board. So if I look on the board,

00:47:09.080 and you look on the board, and we both see that there aren't three of a suit that's on the board,

00:47:13.340 it eliminates the possibility of a flush from the other person's hand. That becomes important,

00:47:18.780 because if I have a hand that is vulnerable to a flush, like if I have three of a kind, or two pair,

00:47:26.400 or something like that, that now increases how I think about the strength of my own hand.

00:47:33.500 And you get to look at your cards as soon as they're dealt, the two?

00:47:35.800 As soon as they're dealt.

00:47:36.640 You get to see your private cards.

00:47:37.960 Right. So you know what your private cards are the whole time. And then these community cards are

00:47:42.840 coming down in the middle, and those community cards are defining for you what you think the

00:47:47.460 other player might have. Now, this is a lot of where the skill in poker comes from, is understanding

00:47:54.340 what those community cards mean. Understanding how they define what the other player has, how they

00:48:01.580 might relate in a logical way to what the other player has, what the probability is that the other

00:48:06.960 player's cards match up in some way to those five cards in the middle.

00:48:10.980 Yeah, it's both, right? It's what do those cards mean about what you have? And what do they

00:48:15.160 not mean about what other players have?

00:48:17.960 Right.

00:48:18.160 And it's interesting, because depending on how many players are at the table, and what's the

00:48:23.380 typical number of players that are at the table?

00:48:25.100 In Hold'em?

00:48:25.800 Yeah.

00:48:26.060 It's generally nine. Sometimes it's eight. The higher the limit, they tend to limit the number

00:48:31.680 of players more. So if you're playing in a much lower limit game, you might see 10 people at a

00:48:37.120 table. If you're playing in a much higher limit game, it's more likely to be eight.

00:48:41.340 Which is still kind of staggering to me, because now, so you have this math of, here's what I have,

00:48:48.800 here's what's common. Based on what's common, what can I impute about everybody else? But now I

00:48:54.900 actually have to pay attention to the behavior of the other people. So I can do the probabilistic

00:48:59.420 calculation broadly. I'm doing two probabilistic calculations. The implications of these cards on

00:49:05.960 me, the implications of these cards on people for whom I don't know the other two cards.

00:49:10.600 But now I have to do eight different behavioral checks.

00:49:14.140 Right. So one thing about the way they relate to your own hand is probabilistic, and one isn't.

00:49:18.860 So the thing that isn't probabilistic is, these relate to my cards, my hand in a certain way.

00:49:23.860 So if I'm looking at the middle hand, like I know what I have, hopefully. But what is probabilistic,

00:49:30.520 and this is what's really important, is what does that mean for whether I have the best hand or not?

00:49:35.020 So that I don't know, because that's relative to your hand, because there isn't an absolute

00:49:39.460 ranking. It's not like, oh, I have a flush, I get to win automatically. Now, that comes up,

00:49:45.740 like sometimes, if I look at the center of the board, I have the very best possible hand that

00:49:51.060 a person could hold. That's very rare that that occurs. Most of the time, I'm somewhere in between

00:49:57.540 the worst possible hand that a person could hold, and the best possible hand that a person can hold.

00:50:02.260 So I have to figure out where I'm sitting on that spectrum, right? And it's probabilistic in nature,

00:50:07.120 whether I'm likely to be beating you or not. And I can think about that in the sense of sort of

00:50:12.940 just base rates. Like if I'm holding this strength hand, how often would I be winning? But then I also

00:50:18.980 have to adjust those probabilities for what you're doing and how you're acting. And how you're acting is

00:50:25.060 going to tell me something. Two people could act in the exact same way, but depending on my model of who

00:50:30.900 they are, it could mean very different things. So I could bet strongly with a hand. And if you call,

00:50:37.000 that could mean that you have a particular range of hands that's X. And if another person calls,

00:50:44.980 their range of hands that they might call with could be narrower or wider. They could be willing

00:50:49.220 to call with either a hand that's much worse than the bottom end of the range of hands that you'd be

00:50:53.340 willing to call there with. Or they could actually be calling with where their bottom range is better

00:50:59.300 than what your bottom range is. As a general rule, does a newcomer to a game have an advantage

00:51:05.260 or a disadvantage? Let's say you and six of your friends have a regular game going and I show up as

00:51:12.080 a first timer. You guys have never played with me. Does that give me an advantage or disadvantage

00:51:16.580 in terms of the point you just made about ranging me?

00:51:19.340 All things being equal. So let's pretend you know nothing about this. Until we play enough,

00:51:24.220 you don't know if I'm good or not. So let's assume I know the fundamentals of the game.

00:51:28.120 I would say all things being equal against the game, you're at a disadvantage because you don't

00:51:33.680 know how to range six people. But me personally, I would prefer to play in terms of my ability to

00:51:40.240 range you. I'm going to be better with somebody I've already played with before. It depends on what

00:51:44.000 question you're asking. And in fact, there's actually been many, many cases in poker where

00:51:50.020 someone has come in and they're doing something that's just really unusual, that is not something

00:51:59.060 that would be expected. And they do really well a certain period of time. And then the market basically

00:52:05.860 kind of figures out what they're doing. Either they adjust or they don't. But I've seen a lot of

00:52:10.880 players come in and win a lot. In some cases, it's just attributable to their short-term bursts of

00:52:16.720 luck. But in some cases, they're actually doing something really unexpected. And people are

00:52:22.140 actually miss ranging them. This usually is because they're playing more aggressively with hands that

00:52:27.840 people normally wouldn't play that aggressively with. And then people sort of figure it out and

00:52:32.340 they adjust. And very often that person starts losing after that. Sometimes they adjust and they

00:52:37.080 do other things. But often it has to do with this problem of sort of what do you have? For example,

00:52:42.140 like in cycling, a totally unknown cyclist can win a stage of a race because they lead out,

00:52:48.100 they break out. And the peloton sort of says, well, there's no way this person can do such and such

00:52:53.320 because we don't know anything about them. And all of a sudden they let them get away. Now they're not

00:52:57.720 going to be able to do that the next day because all of a sudden the peloton is going to say, well,

00:53:00.560 wait a minute, wait a minute. Now we're going to treat you like we would treat anybody who has the

00:53:03.980 potential to win and your moves now fit into a different frame. Right. Exactly. That's exactly

00:53:09.240 what happens in poker. The first thing that you're trying to do is figure out, I know sort of in an

00:53:14.860 absolute sense what I have, but I don't know where that sits relative to what you have. And then once

00:53:21.280 I sort of figure that out and that's probabilistic in nature, I now have to figure out for you what's

00:53:26.920 the best line of play for me. And tell me how the flow of the game goes from here. So the five

00:53:31.040 cards common, we each have our two. How does the betting work and what changes with the showing

00:53:37.140 of cards or movement of common cards? So you start off, you get dealt two cards and there's a round

00:53:43.000 of betting and you can choose at that point. And this is actually a really important concept in

00:53:47.580 poker. I actually tweeted about a little while ago because I think people should be talking about

00:53:51.620 it more, which is you're totally allowed to fold. Remember when Steve Dubner and Stephen

00:53:56.040 Leavitt wrote about this in Freakonomics, like the benefit of quitting. We do not spend enough

00:53:59.960 time encouraging people to quit. No, we spend a lot of time encouraging people to stick to it.

00:54:04.880 And it's actually for everything that you stick to, there's huge opportunity costs involved,

00:54:09.800 right? There's all the other things that you could have done. And if you're sticking to something that

00:54:14.000 you're negative expectancy to, and I'm not saying money, I mean, I'm saying you might be negative

00:54:18.000 expected happiness to it or negative expected health to it or negative expected whatever that,

00:54:23.560 and you're giving up the opportunity to go do something. I mean, I'm a big fan of quit fast.

00:54:28.060 So what I like, my theory is do lots and lots of things that are basically a free roll where

00:54:33.560 there's very little downside to it. Go try out piano, right? It's like a little bit of your time.

00:54:39.300 You can't die doing it. Just go see if you like it, but then quit fast. Just quit fast and move on

00:54:45.800 to something else. And then if you find out, oh, I really love this. I really want to spend my time

00:54:49.700 do it now stick to it. So I'm a huge fan of that. But in poker, this idea is very natural for a poker

00:54:55.860 player. Cause you're doing a lot of the quitting part. You're just saying like, no, this isn't a

00:54:59.580 hand that I mathematically want to play. I don't think I realized that. And that actually might

00:55:03.820 answer this question, which is one of the things I've never understood about gambling in general

00:55:08.240 is you lose much more than you win, presumably. No, not in poker. Ah, because of what you just said,

00:55:14.420 because you can cut your losses. Okay. That answers the question. Cause my question was going

00:55:17.580 to be, I think it answers my question. Given the power of loss aversion, how the hell do people

00:55:23.860 keep playing? Because you'd think if you only win once out of every 10 hands, that's nine

00:55:30.600 experiences that are each more painful than the pleasure of the win. But of course you don't

00:55:35.840 view those nine as losses. You just view them as non wins. You do lose real hands. I mean,

00:55:41.280 you get to the end of a hand and you lose real money. But a lot of the times, if you really feel

00:55:45.420 like, you know, the table, well, you know, the dynamic, well, you just cut your losses and you're

00:55:49.060 like, I'm going to let these two guys fight it out over here. But right. Exactly. And I'm going to

00:55:52.480 only go to the mat when I really have a shot to win. And presumably losing that still falls into

00:55:57.320 the loss aversion pain. By that point, the pre-test probability is such that you have a better shot.

00:56:02.440 So it comes back to this kind of variable ratio reward schedule. You're winning enough big pots.

00:56:09.880 If you're playing this particular form of poker, which is limit, let me just say something about

00:56:13.780 poker. It comes in two styles. One is limited betting. So in limited betting, it means that,

00:56:20.200 for example, if we're playing, maybe we have to bet in like $10 increments. So I can bet $10. And

00:56:24.820 then if you want to raise the bet, you can make it $20. And then if I want to raise it, I could raise

00:56:29.260 it to 30. In other words, we're raising in $10 increments and we can't do anything more than

00:56:33.260 that. What comes along with that is there's limited exposure to the person, but it obviously limits what

00:56:38.620 I can win from you on a particular hand. And then there's no limit, which is what you really see on

00:56:43.400 television, which is the super exciting. Someone shoves a million in chips on.

00:56:47.900 Like when I think of poker, I think of rounders. I'm sure there's been a million movies, which is

00:56:51.620 no limit, no limit and snapping Oreos. I want to be around John Malkovich. I want Oreos and I want

00:56:58.580 to hear a Russian accent. Like that's all right. Anytime that movie's on anywhere, like if I turn

00:57:03.480 on Netflix and somehow it pops up, I'm watching it. Right. Yeah. That's no limit, which is like

00:57:08.080 that super sexy. Oh my God. It's incredible. I'm going to lay everything on the table. Can I bet my

00:57:13.600 lung, you know, but there's also something called limit. So in limit, a really good limit

00:57:20.100 player at the end of an eight hour session, a very good limit player is going to have won

00:57:26.700 about 56% of the time. This is a really, really excellent player. Now go talk to anybody on

00:57:32.600 wall street. They can get a hundred billion AUM. If they say for every bet has a 56% chance

00:57:40.320 of winning, right? I mean, this is a huge ad. So I'm saying this is a really, really,

00:57:43.720 really good limit player. What that means though, is that the person that they're playing against

00:57:47.860 is going to win 44% of the time. So think about how much reinforcement they're getting. So sometimes

00:57:52.980 they're going to win three sessions in a row, right? I mean, 44%, that's a lot. This is a great

00:57:57.580 player. That's hard to believe. Even when you bring it back to the context of what we keep using as a

00:58:02.020 great comparison, which is sports. Think about this. If the New England Patriots played, let's see,

00:58:07.000 who was the worst team in the league last year? I don't even remember. If you put the best team

00:58:10.880 against the worst team, it would not be 56, 44. It wouldn't even be close. It would be 85, 15, or 90,

00:58:17.440 10, even in professional sports. So that tells you how much narrower the gap is here in poker.

00:58:24.100 This is specifically in this limit version of poker. Now in no limit, you can get these bigger spreads.

00:58:29.900 Ah, the 56% is dollars, not number of times in which a person wins.

00:58:35.780 56% of the time is at the end of an eight hour session. How many times have I won?

00:58:40.160 Have you won? Okay. At the end of that session. So over eight hours. Now, just to allow you to

00:58:45.780 understand, it's just a matter of the number of iterations, because sort of in the same way that

00:58:50.960 if the Patriots are 56% against another team, if they play enough times, they're 100% to have won the

00:58:58.160 series. So a poker player who's going to win 56% of the time at the end of eight hours, at the end of

00:59:03.340 1500 hours will approach to 100%, that they'll be in the money at the end of the year. And this is

00:59:08.300 in this limit version of poker. Now, when we get to no limit, basically what's happening is that

00:59:12.900 you're realizing, you're more likely to realize you're- You're going to see huge volatility and

00:59:18.320 you have enormously asymmetric outcomes at times. Right. So here's an interesting thing,

00:59:22.640 actually, because of that, is that when I started playing poker- I love that we haven't even got to

00:59:27.420 you playing poker yet, but that's okay, because we're going to get there.

00:59:29.680 When I started playing poker, the pool of players was small. So it wasn't on TV. It wasn't on the

00:59:36.960 internet. There just wasn't this huge market of people who were like, oh, I'm going to go and play

00:59:42.180 poker. So what that meant was that this limit version of poker was really what was being played

00:59:52.140 in casinos by professionals. And the reason was that when you had someone come in who was not

01:00:00.360 such a great player, making sure they were getting that frequent reinforcement was really, really,

01:00:06.180 really important. So you can think about it this way. If you have kind of an unlimited supply of new

01:00:12.160 players coming into the game, when you come in, I should just take all your money right then,

01:00:16.500 because who cares if you get depressed and go away? Who cares if I hit you over the head and it 0.97

01:00:22.380 plays more like chess? And you're like, well, I'm not going to play anymore because-

01:00:27.440 It's not even close.

01:00:28.120 It's not even close. That's ridiculous because there's literally 17 people standing behind you

01:00:33.280 waiting to take your seat. So now what I can do is I can just maximize in that moment. I can just

01:00:37.800 take as much of your money as quickly as possible because there's going to be somebody else willing to

01:00:41.680 come into the game. When I first started playing, that was not true. When I first started playing,

01:00:45.860 if someone came in the game who was really playing at negative expected value, you didn't want to

01:00:50.780 take a sledgehammer to them. You wanted to make sure that they were enjoying themselves, by the way.

01:00:57.020 And so playing these limit games, which were kind of like a softer way that the edge played out over

01:01:02.620 a longer period of time, was kind of the way to go because then they won enough. It was like they

01:01:08.860 won enough of the time to feel pretty good. And the thing is that I think that people focus when

01:01:14.460 they're thinking about poker on the expected value being about money. And when this person

01:01:21.160 came in and they were losing money, it wasn't that awful for them. When somebody goes to see a sports

01:01:25.860 game, they play money. They get something in return for it, right? They get to watch a good game.

01:01:30.380 When somebody goes to a restaurant and they order food, they're giving money.

01:01:36.940 Yeah, there's a guarantee expected loss.

01:01:38.980 Right. But for them, what they're getting in return is of value. So generally, when these

01:01:44.380 people would come into the game who would be playing at negative expected value, if you only

01:01:48.020 thought about money, I think that most of them were actually playing at positive expectancy when

01:01:52.820 you looked overall at what their overall experience was. Now, the thing is that no matter who you are,

01:01:57.480 it's not fun for somebody to hit you over the head with a sledgehammer. Like here, just give me all

01:02:01.660 your money. It happened in two seconds. Now go away. But when they would come in and they'd get

01:02:07.580 to play with really good players and they'd be learning the game, which is exciting. And they'd

01:02:12.440 feel like, you know, I'm playing at table one at the Bellagio or Aria or whatever. And they're winning

01:02:18.640 enough to feel good. They're probably learning. And this is entertainment for them. Now it's no longer

01:02:26.320 zero sum anymore. It's like, is it zero sum as far as the money is concerned? Sure. But it's not zero

01:02:32.260 sum overall when you look at like, what's the scope of what people value? Because there's all this

01:02:36.480 entertainment happening as well. And I remember there was this guy, I won't say his last name,

01:02:40.560 but this guy named Jay, who came and played, he came in the late 90s. And he, you know, he's probably

01:02:46.100 mid to late 70s. He had made a bunch of money on Wall Street, moved to Las Vegas to retire. His health was

01:02:55.240 okay. His mind was super sharp. And I think that he just sort of decided this is what he wanted to do

01:03:02.060 for his retirement. He had a ton of money. And so he started playing in these high limit games with

01:03:07.120 the high limit players at that time. And it was a small group. And he'd come in and play. And the

01:03:13.620 whole group would break for dinner. And we'd all go with Jay to go eat at a restaurant in the Mirage.

01:03:20.600 It was at this time, this was before the Bellagio was built. We'd all go eat dinner. And he would

01:03:25.020 sometimes have parties at his house. And we'd go over to the party at his house, or he'd get

01:03:28.820 invited to somebody else's house. And this was his social life. And he was losing a lot of money,

01:03:34.060 but he was happy. He was having fun. He had this whole group of people. And he got to have this fun

01:03:40.120 conversation. And he got to learn this game and so on and so forth. And I think this is kind of how he

01:03:44.900 spent the last four or five years of his life. And he was for sure winning to that. So I feel like

01:03:50.620 people sort of think it all sounds very cutthroat. But it's not. And this is something that certainly,

01:03:56.160 I think that the players in the 90s, for sure, were really thinking about, which is what's the

01:04:01.200 value add? How are you bringing value for this? We know how they're bringing value to us.

01:04:06.180 Wow, that's so interesting. I wouldn't have guessed that. But that's a nice symbiosis.

01:04:09.520 There's a really, really famous player named Chip Brees. He sadly passed away about,

01:04:13.920 very unexpectedly, he was young. Passed away probably about 15 years ago now. Maybe the

01:04:19.540 greatest poker mind ever. I mean, certainly one of them. And the thing about Chip was he just made

01:04:25.400 everybody happy around him. He made a ton of money playing backgammon. He made lots of money playing

01:04:31.040 poker. And he had the superpower of these people who were very negative expectancy, if you were thinking

01:04:36.600 about money, would be calling him up like dying to play with him. Because he was nice. It was just a

01:04:41.960 really good time. I think about it as the same thing. Why do you go to a movie? Well, okay,

01:04:46.760 if we just think about it as money, it seems like a losing proposition. But of course, that's not what

01:04:51.820 we should be really thinking about that as. And so I think that that gets lost a little bit

01:04:56.080 as the game grows and becomes really, really, really big. And so if you don't have a particularly

01:05:02.940 good time, and you're sad, and you leave, there's somebody else just to take your place.

01:05:09.100 And I think that that interesting aspect of how it really wasn't very zero sum at all

01:05:14.820 kind of goes away. I think it's come back a little bit now because the poker economy has

01:05:19.680 contracted. And I think they're sort of coming back to that idea of how does this become fun

01:05:24.880 for everybody? It's a really interesting aspect of the game back then.

01:05:28.680 So when did you figure out you had a knack for this?

01:05:30.780 It's hard to say, because I feel like it's all relative. You know, I retired in 2012. And

01:05:35.100 pretty much anything else that I can think of that's competitive. Poker's really changed the

01:05:41.600 ability for people to be able to run batches of hands, for example, after the fact is sort of

01:05:48.200 think, oh, here is the situation that I was in. Now I'm going to go and I'm going to run this and

01:05:52.600 sort of see what the right answer would have been, so that I can take that into the next time I play.

01:05:57.200 The kind of analytics that people have, the ability for them to discover what the equilibrium

01:06:02.060 point is on a particular decision or whatever. It's just beyond what I can even comprehend.

01:06:08.040 So I left before I would have had to put that work in. If I did put the work in under those

01:06:14.400 conditions, I'm not sure that I would do well. I mean, I know that if I sat down today without

01:06:19.060 doing that work, I would get completely crushed. So in that sense, you can look at me and say,

01:06:23.700 well, I didn't have a knack for it. If I tried to play against today's players who know the things

01:06:28.920 that they know and are able to run the kinds of simulations and analytics that they can run now

01:06:34.220 and they're understanding the mathematics of the game are and sort of what the optionality is.

01:06:39.740 So you're saying that a professional player today will take every hand of a game and they'll have

01:06:44.620 the simulation after the fact of that hand?

01:06:47.020 Well, not necessarily every hand, but they can pick hands and they can actually run the same way

01:06:51.200 that you do it in baseball.

01:06:52.560 Well, I was going to say, this sounds a lot like Moneyball, which is the advent of analytics to

01:06:57.060 the game.

01:06:57.780 Right, exactly. So they can sit there and they can say, they can just set the parameters. If I

01:07:02.000 am in a particular situation where I think that I have to bluff and if I give you this range of

01:07:07.680 hands, how often do you fold? If I give you this range of hands, how often do you fold? And they

01:07:11.960 can figure out exactly at what point, if I bet a particular amount, am I winning enough to that bet

01:07:19.180 that I can now try to bluff in that situation that maybe I would have never thought in a million years

01:07:24.100 was a situation that I would be winning to if I were to bluff there because I just, I didn't have

01:07:27.900 the data. So I feel like knack is such a strange word because I feel like it's knack relative to

01:07:35.220 what?

01:07:35.920 Well, to your peers at the time.

01:07:37.620 Yeah. So those peers changed. So I started playing in Montana.

01:07:41.900 What year is this?

01:07:42.940 Oh my gosh. I started playing, I turned professional in 94. So it's right around that time. And I was

01:07:48.660 playing against people in Montana who a lot of them were retirees who were, this is kind of what

01:07:54.380 they did. Some people have their bridge game or some people are playing golf or some people are

01:07:59.280 playing poker and that's kind of what they wanted to do. So while I think that pretty much everybody

01:08:03.880 who plays poker would like to get better and enjoys the game more, some people are more hungry

01:08:09.920 to do that than others. And people are getting different things out of the game. And the people

01:08:14.520 that I was playing with were not thinking, let me become a world-class player. At that time,

01:08:21.800 particularly because I had really great mentors, my understanding of the mathematics of that game

01:08:27.560 was so beyond their understanding. I don't think that was part of their enjoyment. So the day that

01:08:33.640 I stepped into that room, I had a knack, certainly compared to the people that I was playing against.

01:08:39.380 I started playing two years before that in that game and that was sort of where I was learning.

01:08:43.080 And then in 94, I decided I was a professional poker player and I moved to Las Vegas and I started

01:08:49.160 playing there. And this is how many years after the game in New York? Or was it only your brother

01:08:53.060 that was playing? Only my brother was playing. Yeah, I never played in New York. I don't think I ever

01:08:56.860 played a hand of poker in New York. So you get to Vegas in 96? No, I got to Vegas in 94. Oh, that was

01:09:02.720 the declaration of running pro. Okay, got it. Does it become less fun when it's your job now? Does it amplify

01:09:07.960 the stakes? Eventually. But no, not at this time. Poker is a really complicated game. Very, very

01:09:14.320 complex. And I know that there's just so much that I don't know about the game. And I'm totally aware

01:09:21.700 that if I tried to sit down and play against today's players, that they would bury me. Because

01:09:27.000 there's this whole layer of the game that I haven't discovered. But while I was particularly in those

01:09:33.080 early years in the 90s, I had started off in Montana playing only limit Hold'em. Now I go and

01:09:41.060 I start to learn more about that game. So I'm starting to sort of peel back the layers of that

01:09:44.880 game. And like most things that you do, as you peel those layers back, what you start to see is that

01:09:50.120 there was a whole bunch of stuff underneath it that you didn't even know existed. Right? So you think

01:09:54.420 that the onion is a certain size, and you peel a layer back and you realize, oh, wait, no, there's like

01:09:58.960 so many more layers under there than I ever could have thought. I was telling this to somebody the

01:10:02.800 other day, my head of research always says, it says, I don't know if he's paraphrasing somebody

01:10:06.780 or if he coined the term, but the further you get from shore, the deeper the water. Somebody

01:10:10.320 asked me a question about a topic. I don't remember what it was. But I said, and I wasn't

01:10:15.080 being facetious, I really mean it. I know less about this subject today than I did 10 years

01:10:19.980 ago. And they said, how is that possible? And I said, well, to be completely accurate, I mean

01:10:23.920 on a relative basis. Obviously, on an absolute basis, I know more than I did 10 years ago about

01:10:28.400 subject Y. But my knowledge of how much broader it is today has dwarfed my absolute knowledge.

01:10:36.800 So on a relative basis, I've gone backwards. Yeah, exactly. That's what's happening. So I'm

01:10:41.800 learning this game, Limit Hold'em, learning things about poker in general. And then I start

01:10:47.040 expanding my repertoire of the types of games that I play. So I start to learn the Omaha games,

01:10:51.880 I start to learn the stud games, I start to learn pot limit and no limit. So those are different

01:10:56.820 betting structures. And I'm immersed. It's like I'm living and breathing and talking poker

01:11:02.500 all the time with peers, with mentors, so on and so forth. So it's not really like a job in that

01:11:11.400 sense. Certainly not during that first eight or 10 years that I'm playing, I'm learning. And every

01:11:19.540 second is learning. So when did you sort of wrote about this in the book, but you also figure out it

01:11:24.800 seems pretty early that there are at the risk of oversimplifying two different types of players

01:11:30.360 there are the players who when they win, it's because of how well they've done things when they

01:11:37.000 lose, it's because of bad luck. And that's basically the narrative. And you seem and again, I don't know

01:11:44.160 how quickly you came to this understanding. But at some point, you seem to come to the understanding

01:11:48.740 of wait a minute, sometimes I'm winning when I make the wrong bet. And sometimes I'm losing when

01:11:55.460 I make the right bet. And that last one, by the way, must be a hard pill to swallow. You end up sort

01:12:00.540 of in your own mind formulating a very common tool we use today, which is this decision matrix of did I

01:12:07.460 use the right process? Yes or no? Did I get the right outcome? Yes or no? That's a lovely two by two.

01:12:14.240 But it's one that you have two squares of that two by two never get talked about, which are the ones

01:12:20.040 that are at odds with each other. Let me really dig into something you just said. So I agree that

01:12:25.260 we're not seeing the whole two by two. Where I'm going to quibble is this. Correct me if I'm wrong,

01:12:31.280 but I think what you said was you could have made really good decisions and had a good situation and

01:12:38.700 ended up having a bad outcome. And that's the really hard pill to swallow. And I actually think

01:12:44.600 it's the reverse. To me, I think that's the story you tell when you're opening your book, which

01:12:48.500 bringing it back to the New England Patriots is when the tables are turned, right? It's the Pete Carroll

01:12:53.120 pass call on the two yard line or whatever in the Super Bowl that gets intercepted, which again,

01:13:01.160 by all metrics, if you looked at this, like you were a robot looking at reams of data, the outcome

01:13:08.240 of that call was positive. That should have been a touchdown. Right. That's a question of resulting.

01:13:13.800 When we're evaluating other people's decisions, particularly people that we don't compare to,

01:13:18.420 particularly in a situation where the decision is complicated, we have this simplifier, the shortcut

01:13:25.080 called resulting. So what we do is we say, if I know what the quality of the outcome is,

01:13:30.260 I can work backwards from that to the quality of the decision. So this is what happens to Pete Carroll.

01:13:35.220 It's the last play of Super Bowl 2015. Everybody expects he's down by four to the Patriots,

01:13:42.300 as always. It's always the Patriots. Back to the beginning of the conversation. At any rate,

01:13:46.600 everybody thinks he's going to hand the ball off to Marshawn Lynch. Instead, he has Russell Wilson,

01:13:50.560 the quarterback passed the ball. This is very unexpected. The math is very, very complicated

01:13:56.280 here. There's some game theory in there. There's options theory, all sorts of things that people

01:14:00.840 are interested in reading about it. They can go look at Benjamin Morris on 538. Suffice it to say,

01:14:05.580 it is a good play. We'll link to all of this in our show notes, by the way, because we love nerding

01:14:09.340 out on this exact sort of background. So he passes the ball. It's intercepted. They lose the

01:14:15.580 Super Bowl this way. Malcolm Butler, my hero. Just, I mean. I can't imagine. Can you imagine

01:14:22.520 what I was doing to my TV? I remember looking at my daughter going, earmuffs. And it was like,

01:14:29.960 fuck, yeah. I mean, I am screaming. My wife's like, all right, I'm going to take the kids out. 1.00

01:14:35.540 I'm like, yeah, yeah, yeah, that's fine. Yeah, exactly. So this is what we do. The thing about

01:14:39.500 understanding the quality of the decision, it's complex. It's really hard. You need to know a whole

01:14:45.560 bunch of different things. You need to know, given the decision that was made, what were the

01:14:50.880 possible outcomes? How often would those outcomes occur? If you think about what are the consequences

01:14:57.560 of each of those outcomes? How much do I prefer each of those outcomes so I can understand what

01:15:01.180 the probability of getting a preferred outcome is? If I have an outcome that I don't prefer,

01:15:06.300 like an interception, like how often does that happen? And then you also have to compare it,

01:15:09.580 do that exact process with the other possibilities, the other decisions that you could have made

01:15:15.340 and now actually compare to see how do I get the highest expected value, the highest chance of

01:15:19.660 actually winning the Super Bowl. And it's all very complicated and involves what's the interception

01:15:24.580 rate, which people don't know off the top of their head. Or you have to think about if there's

01:15:30.160 only 26 seconds left, how do I get three plays off instead of two? And it's very hard. Suffice it to

01:15:35.480 say, if you pass the ball either on the first or the second play there, because it's second down,

01:15:41.160 26 seconds left, one time out. If you pass the ball on either the first or second play,

01:15:46.660 it means that you can do that plus hand the ball off to Marshawn Lynch. If all you do is hand the

01:15:53.180 ball off to Marshawn Lynch twice, you don't get that third play. That third play is incredibly valuable.

01:15:58.480 And the cost of the third play is whatever the interception rate is. And the interception rate

01:16:02.860 there is less than 2%. And there's a time management piece to it as well. A pass that is not caught

01:16:09.200 as an immediate time stop. You could have a bizarre scenario under which you hand it off to Marshawn

01:16:14.700 Lynch. It turns into a bit of a scrum. You have to burn one of your timeouts. Well, you're going to

01:16:19.080 have to no matter what. You only have one left, if I recall. Right. So you have one left. So that's why

01:16:24.060 if you hand it off to Marshawn Lynch on the first two plays, assuming the first one fails,

01:16:30.040 that's the end of your timeout. So because you can get the clock to naturally stop on a pass, that's why

01:16:35.680 if you pass on the first or second play, you can get three plays instead of two. But that's all very

01:16:40.860 complex. So what we tend to do in those situations is we say, that's all really hard. And I don't

01:16:47.300 know what any of that is. And it's all very opaque to me. So if I understand what the quality of the

01:16:53.200 outcome is, was it good or bad in this particular case, it was disastrous, then I know what the

01:16:58.260 quality of the decision is. So that's what's happening in that Pete Carroll thing. That's called

01:17:01.600 resulting. Resulting is a heuristic simplifier. Works really well in chess. If I lose to you in

01:17:07.940 chess, I played worse than you. Works very poorly in poker. If I lose to you in poker, who knows?

01:17:13.800 And certainly very poorly in football. We don't really know, depending on how the play worked out,

01:17:19.600 whether it was a good or bad decision. But what's interesting is that people don't do this resulting

01:17:25.540 thing in situations where they feel like the quality of the decision is known. And I say

01:17:31.520 feel like because in quotes, yeah, right, because they don't necessarily know it. But remember that

01:17:37.020 what I said is resulting is a simplifier, the quality of the decision, it's complicated, it's

01:17:40.920 hard to figure out. So what happens when we feel like we do know the quality of the decision. So I'll

01:17:44.920 give you an example. If I come home one day, and I say to you, I got in a car accident. Do you know

01:17:55.080 if it's my fault or not? No clue. And you won't use the fact that I got in a car accident

01:17:59.880 to work backwards, like you do with Pete Carroll, because you understand that there's certain things

01:18:06.520 that really are knowable about driving. And you can ask me what those are. Did you break the rules

01:18:11.760 of the road? Were you drinking? You can ask me a whole bunch of things. So there's all sorts of

01:18:17.420 places like that, where we feel like we understand what the decision quality is. And so if we have a bad

01:18:22.220 outcome, we actually tolerate it. This occurs all the time in business as well, right? If somebody makes

01:18:28.560 a decision, let's say they execute on a decision that was made in a committee, where the whole team

01:18:35.880 agreed that this was the right way to go. And then it doesn't work out, people aren't like idiot. But 0.99

01:18:43.620 if they do something on their own, that is not something that everybody always does, then all of a

01:18:52.560 sudden, if it doesn't work out, they're really under the gun. So we have this issue of transparency that

01:18:57.140 occurs. So now let me get back to this roundabout way. So first of all, let me just give you a good

01:19:03.100 example of the transparency problem. Okay, bear with me on this thought experiment for a second.

01:19:07.840 You don't have ways, it's broken. And you're driving to the airport with your spouse.

01:19:15.160 Obviously, you have to get there in time. And you get in the car and you go the normal way that you go,

01:19:21.380 that you've always gone. And something happens, there's a big accident on the road. And your car

01:19:28.760 is not moving. And you get to the airport and you just missed your flight. Are you getting blamed

01:19:33.280 for that? Is that your fault? Is your spouse mad at you?

01:19:36.400 No, not in my case. I can imagine in some people's cases.

01:19:38.940 But mostly, no, it doesn't feel like it. Okay, so now you get in your car again, you don't have

01:19:42.720 ways, whatever your phone's broken, you get in your car, and you announce to your spouse,

01:19:46.860 I have a shortcut. I was just talking to Morgan over there. And they told me about this great

01:19:53.920 shortcut. So I'm going to take the shortcut because I've checked it out. I looked on a map,

01:19:57.920 it's better. And now you go to the airport, and there is an accident, car is not moving,

01:20:03.440 you make it to the airport, and you've just missed your flight. Is your spouse mad at you?

01:20:08.140 Again, mine would not because she's superhuman. But yes, I think many people in that situation are

01:20:14.060 absolutely getting yelled at. By the way, that's a superhuman spouse. It's like, 1.00

01:20:17.700 I cannot believe your stupid shortcut. So this is one of those cases where, 1.00

01:20:21.960 so we have this come up all the time. And when it comes up in so many different ways,

01:20:27.380 it comes up in medicine all the time. Because that's such a great example, right? If Marshawn

01:20:31.820 Lynch had been handed the ball, and he had fumbled the ball, same exact outcome, people would be pissed

01:20:38.000 at Lynch. Nobody would be pissed at Pete Carroll. That is exactly right. And they might not even be

01:20:42.360 pissed at Marshawn Lynch. They might just think that the Patriots line is too good.

01:20:46.760 Yeah, the safety or whoever came and stripped the ball out was just exceptional.

01:20:50.500 Right. And why is that? What's the difference? Because in both cases, you fail.

01:20:54.180 Expected versus not expected.

01:20:55.540 Expected versus not expected. And then this comes up, I guarantee this comes up in your business all

01:20:59.540 the time. It comes up in your personal life all the time. It's one of the reasons why,

01:21:04.300 for example, if someone's really unhappy in their job, they won't be aggressive enough about going and

01:21:10.960 finding another position. And the reason is that the unhappiness in their job is now sort of become

01:21:17.180 expected. Like they're not really blaming themselves for that. But if they go and find a

01:21:20.560 new position, and they're not happy there, they're going to feel like it's their fault. As an example,

01:21:24.600 this happens in business strategy. This is the way that teams end up operating, so on and so forth.

01:21:29.560 Okay, so this becomes a really big problem. So every human being understands this difference between

01:21:34.720 kind of transparent and opaque decisions. Now, here's my question for you. What decisions do you

01:21:39.860 think are the most transparent to you? Your own. These have the most transparency to us.

01:21:46.860 So what happens with our own decisions is that we allow uncertainty to bubble to the surface as the

01:21:54.960 explanation for a bad outcome more often than we would for somebody else's decisions. Why? Because

01:22:01.280 we feel like we know what the quality of our decision is, number one. And number two, we don't

01:22:05.780 really want to question the quality of our decision. So you can think about that sort of in general,

01:22:09.920 do you hand it off to Marshawn Lynch or do you pass it, is a question of when are we allowing

01:22:14.280 uncertainty as the explanation for a poor outcome. So it's innovative, unexpected, opaque, all of those

01:22:21.960 things. We don't allow the uncertainty in the door. But when we feel like we know, because obviously

01:22:26.900 you're supposed to hand it off to Marshawn Lynch, now we let uncertainty into the door. We do this for

01:22:31.040 ourselves as well. So now this brings me full circle to this thing about when we think about

01:22:35.980 this matrix and you say, well, you might make a really good decision and have a bad outcome.

01:22:42.720 That's the most painful, you said. By the way, that's just my subjective overlay. I don't know

01:22:48.020 that there's any decision-making expert out there that would argue one way or the other. Well, I think

01:22:52.240 that people do think that. So I just want to sort of pick at something. So let's just start again.

01:22:56.220 So what you said was there are these two quadrants, right? That we would think of as discordant.

01:23:00.860 Right. So one of them is you make a bad decision. But you get the right outcome. You get the right

01:23:06.540 outcome. And then you said even more painful is you make a good decision and you get a bad outcome.

01:23:11.580 Now, what I think is interesting is that I would actually argue that that doesn't have a lot of

01:23:17.540 pain associated with it. And let me explain why. Imagine you're looking at this in the aftermath of

01:23:23.120 the outcome. Okay. So in one case, you've gotten a good outcome. In one case, you've gotten a bad

01:23:27.240 outcome. So let me ask you to do the thought experiment. Where do you want to dig in more?

01:23:32.180 If you get a good outcome, are you looking to dig in to try to find out if you made a bad decision that

01:23:38.140 resulted in that good outcome? Or are you just being like, good outcome? Yay me. But if you have a bad

01:23:43.160 outcome, are you eager? Honestly, I think it depends. Like using sports as an example, I think is a

01:23:49.860 different example versus in medicine. So I've talked about this before on the podcast. In medicine,

01:23:54.420 there's a, or at least in surgery, there's a conference that's typically done called the

01:23:58.900 morbidity and mortality conference, where you, as the name suggests, every week you discuss all the

01:24:05.280 morbidities and all the mortalities of the previous week. And so that is an objective definition. A

01:24:12.480 person can basically arrive dead in the hospital from, let's say, a gunshot wound. They still get

01:24:19.560 presented in the conference, even though, let's say you did nothing. They literally showed up in

01:24:24.540 the emergency room, muted CPR for a minute. They were all basically already dead in the call time

01:24:28.020 of death. That still shows up, even though there's no blame to be assigned within the confines of how

01:24:32.600 medicine was practiced. So those ones don't get a lot of discussion. The ones that get a lot of

01:24:36.920 discussion are you made a decision that seemed like the right decision at the time, and it seems

01:24:43.820 defensible, but the outcome was bad. Now, of course, as you've alluded to, these are so complicated

01:24:49.840 because there's still the luck component. There's all these other things that go into it. So I guess

01:24:54.240 if your question is, where do you learn the most versus what causes the most consternation? Those

01:25:00.560 aren't necessarily the same, are they? The point that I'm making is not so much about where do you

01:25:04.140 learn the most? Because I think you learn from all four quadrants, and it's really important to learn

01:25:09.240 from all four quadrants, right? I made a good decision. I got a good outcome. I'd like to

01:25:12.460 understand when I have made a good decision that has a really good expected value, and I want to be

01:25:19.000 able to see that. I want to understand when I made a bad decision that had a bad outcome, and where I

01:25:23.340 had negative expected value, and then I can understand that so I can adjust my decision

01:25:26.900 making going forward. And likewise, I'd like to understand when I made a bad decision that had a

01:25:30.480 good outcome and vice versa. What you just described is actually the whole shebag in terms of the 1.00

01:25:36.600 reinforcement of this bad kind of thinking. Because what I'm guessing does not occur every week,

01:25:42.460 is the, wow, this patient did way better than expected. But we made the wrong decision. Let's

01:25:48.120 have a conference about that. So let me give you a super simple example to try to get at this,

01:25:52.680 because I think that this is actually a really big problem, particularly as I feel like now people

01:25:59.140 have now understood the importance of process language. So if we think about any business movie

01:26:06.480 from the 1980s, they're like, I'm results driven, right? Now, now you sort of look back at that, and you're

01:26:12.480 a little horrified, because I think we all understand that we don't want to be results oriented, we kind of

01:26:16.640 get that that's a problem. And we'd all like to aspire to be Sam Hinckley and trust the process. And we know

01:26:22.400 we're supposed to say process, process, process. But here's the problem. So now let's do this as a two by

01:26:27.700 two. Okay, you have a status quo decision, the expected decision, a consensus decision. And you

01:26:37.780 have a good outcome. Is anybody hailing you as the biggest genius of all time? No, but they're saying

01:26:44.160 good job. You have a status quo decision that has a bad outcome. Is anybody excoriating you? No,

01:26:52.840 they're saying tough luck. What could you do? You have an unexpected, an opaque, a non status quo,

01:27:00.820 a non consensus decision that has a good outcome. And they're hailing you as the biggest genius of

01:27:07.820 all time. You have a non status quo decision that has a bad outcome. And everybody's calling you an

01:27:14.400 idiot. So people understand this. So now let's think about what does that do to somebody's decision 1.00

01:27:20.360 making? Well, first of all, they want to avoid the bad outcome. This is where loss aversion comes

01:27:25.620 in, right? We don't want to be the idiot. That's the one quadrant that you want to avoid, right? 1.00

01:27:29.860 Well, in general, we want to avoid the whole right side of the table, because the right side of the 0.98

01:27:35.180 table contains the idiot section. So let's think about how can we avoid the whole right side of the 0.88

01:27:40.760 table? Well, one way we can avoid sort of the bad outcomes altogether is to kind of fall down on the

01:27:47.760 not deciding thing, just sort of not doing anything that's going to cause a big loss, or a big win.

01:27:56.300 Either you could do it by omission, you could sort of not act, you could make very, very low volatility

01:28:01.840 choices, there's things that you could do there to play these strategies. The other thing that you can

01:28:07.300 do, so you can avoid deciding at all, or you can make sure that you're making things that don't really

01:28:13.620 create big losses, so you're not going to end up in the room. But the other way to deal with that

01:28:18.720 whole right side is to focus on the lower right quadrant and say, I never want to be the idiot in 1.00

01:28:24.060 the room. And so therefore, I'm going to make status quo decisions. So let's think about this. For 0.99

01:28:30.020 example, it doesn't matter if 80% or so of ear infections are viral. If a patient comes into my

01:28:36.680 office, I'm giving them antibiotics, because I don't want to get yelled at. If this happens to be the time,

01:28:41.580 I want to be able to cover my butt. And that way, nobody can yell at me. So now let's think about 0.96

01:28:47.120 this. So what ends up happening is that we want people to be thinking, I just want to make the

01:28:53.680 best decisions. And I don't really want to be afraid of the bad outcomes. I particularly don't

01:28:59.560 want people to be afraid of this idiot quadrant, because we would like people to be innovative. 0.99

01:29:04.260 And we'd like them to be thinking outside the box. And we'd like them to sort of push against what the 0.99

01:29:08.480 status quo is, because that's how obviously, we move forward as a society as a business as an

01:29:13.920 individual. But this is what happens. Everybody has a morbidity conference. So even if in the

01:29:21.480 morbidity conference, you're talking about process, you're in the room because of morbidity. So

01:29:27.440 everybody knows I get in the room because of something bad. So here's a simple thought experiment

01:29:31.960 that I use. Imagine that we're investing in real estate, like we have a real estate investing group,

01:29:37.500 and we have a model of the market, obviously, and we have limited resources that we can deploy.

01:29:43.560 And so we invest in a particular project, and it ends up doing 15% worse than we expected. We're all

01:29:52.740 in a room. Now, we might be in that room talking about our model. True. We might be in the room talking

01:29:59.920 about our decision process. But we're having a long meeting in a room about the fact that the project

01:30:05.940 did 15% worse than we expected. Now we do the exact same thing. We invest in this particular project,

01:30:11.720 and it does 15% better than expected. Is the same conversation happening? No. So I don't care how much

01:30:18.460 the word process comes out of your mouth. Everybody knows they're supposed to be afraid of bad outcomes

01:30:22.900 then. And here's the really big problem. And it's true, we can talk about it in terms of medicine as

01:30:28.700 well. But this particular thought experiment that has to do with the real estate, it matters just as much

01:30:33.680 as if you've overestimated the market, which is how you'd end up 15% on the downside than if you've

01:30:39.400 underestimated the market. Because your goal is to efficiently allocate your resources within the

01:30:46.260 market that you're allocating the resources in. And if your model's wrong, I don't care if it's wrong,

01:30:51.840 if it's underestimating or overestimating, that's a problem for efficient allocation of resources.

01:30:56.840 Not only that, but when it's 15% to the good, it's a signal that there may be risk that you didn't

01:31:02.960 identify. So for example, you could have the mean right, but you may have the variant. That's a

01:31:08.820 really big problem because that has really big implications for what percentage of your bankroll

01:31:14.040 or of the total money that you have, the total capital that you have that you're supposed to

01:31:18.180 deploy to that particular project. That's a very important point, by the way, which doesn't,

01:31:22.340 I think, get talked about enough even in that circle, which is people are very fixated on return

01:31:27.620 on capital, but they generally aren't as appreciative of risk-adjusted return on capital.

01:31:32.600 So Rayrock versus Rock is this concept that seems to be missing from the way a lot of people think

01:31:38.380 about that. So it's good that you brought that point up because there could be a number of problems

01:31:42.620 with that issue. Not just maybe you didn't allocate enough, maybe you allocated the right amount or too

01:31:47.860 much. It didn't appreciate what was at risk. Right, exactly. So we want to be seeing those signals,

01:31:52.780 right? Because we'd like to understand it if we underestimated the vol. And in both cases,

01:31:57.180 your model could be totally right and you can just be a couple of standard deviations away.

01:32:01.180 There's a variety of reasons on that particular time that it could have come in low or high that

01:32:06.680 don't have anything to do with your model. But you kind of want to dig in either way because it's

01:32:11.180 something unexpected happened. The other thing, by the way, that happens is that when you have a

01:32:16.920 downside event and you do get in the room, people rarely ask, should we have done worse?

01:32:20.980 They ask, could we have done better? And now again, you're reinforcing for people,

01:32:25.300 I care when things go badly. When things go badly, I want to avoid them. Now, poker happens to be this

01:32:31.360 very pure place where should I have done worse, regardless of whether I won or lost, is actually

01:32:37.660 a very important question. Because sometimes you have a hand, I'm thinking about what you have,

01:32:44.480 and let's say that I put you in a range where you're very weak and I feel that I'm very strong in

01:32:49.820 comparison to you. So because you're weak, I'm playing the hand, we would call it slow or small,

01:32:56.100 where I'm- You're trying to draw as much out as possible.

01:32:58.500 Right, which means that I'm not betting really big. Now at the end of the hand, you hit a lucky

01:33:02.020 card at the end of the hand, the very last card. I end up losing the hand, but when we turn the hands

01:33:06.800 over, it turns out that you were much stronger than I thought. In this case, I should have lost more

01:33:12.700 money. Because yes, you hit a lucky card, but I should have been extracting much more money from

01:33:18.460 you because you would have tolerated it had I had your hand right. Now it may turn out that there was

01:33:22.880 no way for me to get your hand right, but I'm supposed to explore that. Because now what I know

01:33:27.700 is that if I knew what your cards were, I would have lost a lot more money than I did. That I actually

01:33:33.260 played the hand very poorly for what the hand was that you actually had. I actually underplayed my hand.

01:33:38.980 And that story, that example you use, I think to me is one of the best examples of why the title of the

01:33:44.780 book makes sense, which is, this isn't a book about poker, actually. It's a book that uses poker as a

01:33:51.780 model system. So for example, let's talk about biology. Why do we do so much biology in C. elegans,

01:33:58.260 a worm, or in this type of a mouse, or this yeast? Do we care that much about mice and yeast and worms?

01:34:04.360 No, they are model systems that allow us to do things in cleaner ways. And I think that's sort

01:34:12.240 of the beauty of poker, which for the listener, they might think they're talking about poker so

01:34:16.620 much. Why? Well, I think you have to understand enough about poker to understand what you just said.

01:34:21.560 Because what you just said is, it's not often in life we will get that much clarity after the fact,

01:34:27.740 but we can still bring that level of critical thought to our decisions. To me, that is the,

01:34:34.820 if you remember nothing else from this podcast, this is the part I'd want someone to remember.

01:34:39.120 I think of poker as the C. elegans of decision-making in life. That's going to be my...

01:34:43.800 That's great. And we can think about this on the winning side as well, right? Like,

01:34:48.340 I could win a hand from you where I really should have won a lot less. Again, if I had had perfect

01:34:57.880 information, I shouldn't have actually won as much as I did. Because for example, I could misread your

01:35:03.580 hand and I could play a hand really strongly. And then I can be the one who hits a lucky card.

01:35:08.900 And actually, if I had perfect information, I would not have ever put that much money at risk.

01:35:14.180 So we want to explore whether the outcome is good or bad. We want to have equanimity toward the

01:35:20.220 direction that we're exploring. Could I have gotten even more out of you? Could I have done

01:35:24.100 even better? Could I have done worse? And the other thing that besides that direction, by the way,

01:35:27.920 is we want to think orthogonally. Like maybe you lost for reasons that had nothing to do with your

01:35:31.820 decision process, or maybe you won for reasons that you totally didn't predict. This happens in

01:35:36.960 financial markets all the time. You invest in a, say, a company because you think that a

01:35:43.440 particular product of theirs is going to do gangbusters or something. And then that product

01:35:48.520 fails, but they happen to acquire another company. By the way, don't take credit for that if that

01:35:52.780 wasn't in your model. But you happen to have won. So we want to be thinking up, down, orthogonal in all

01:35:57.840 these directions. So now here becomes the problem. We know that people don't like to be in this bottom

01:36:04.540 white corner. They don't like to be in this place of, okay, something really bad happened. And if they do

01:36:10.720 get in that something really bad happened place, they want to be able to say, but it's not my fault.

01:36:16.600 It's a bad luck situation. It's like, well, my decision process was good and everybody had

01:36:20.900 consensus. And I handed the ball off to Marshawn Lynch. What could I do? That was what I was supposed

01:36:25.480 to do. So you're driving people into this place where they're very likely to be trying to build

01:36:31.080 consensus, not necessarily of the good kind, right, of the false kind. They may be using data not to find

01:36:37.080 the truth, but to tell a data story that supports them in case they happen to get into the room.

01:36:42.240 So we really like to say the data told us so. They may just be doing what everybody has always done.

01:36:47.380 They may be slow to decide because they need buy-in or the right kind of sign-off too often,

01:36:52.000 or they need a higher level of certainty than they really should in order to make the decision. All

01:36:56.580 sorts of bad things come out of this. And what really ends up coming out of it is that people tend to

01:37:01.020 move slowly. They tend to default toward omissions versus commissions. In other words,

01:37:05.960 not deciding versus deciding. And they tend not to like to innovate too much.

01:37:10.900 Now, my guess is there's a different propensity for this across different fields. So for example,

01:37:16.460 have you ever had a chance to talk to Pete Carroll?

01:37:18.380 I haven't.

01:37:19.220 Just thinking about everything you said, it would be very interesting to have that discussion with

01:37:22.740 him and say, it'd be hard for him to do this, but go back in time to before you made that decision.

01:37:27.860 I'm pretty sure the only thing Pete Carroll was thinking about was winning. I can't imagine

01:37:32.940 there was even an ounce of thought in his mind that said, whatever decision I make,

01:37:39.220 I want to make sure it's defensible at the press conference. I just can't imagine he would have

01:37:42.460 thought that, right?

01:37:43.140 I agree. And I think that's what makes Pete Carroll such a great coach.

01:37:46.180 I was just about to say, but wouldn't you see more Pete Carroll-like thinking in something like

01:37:50.660 sports? Or is it less about sports and more about being the head coach versus the defensive

01:37:56.660 coordinator who in the end is still making a very important decision, but answers to a

01:38:03.000 coach? Of course, the coach answers to the owner, but I'm not sure I'm articulating the

01:38:06.360 question well, but...

01:38:07.240 No, I actually think that you are. So first of all, we know across the NFL that the slowness

01:38:11.360 to adopt the analytics has been remarkable. And that's because in this particular case,

01:38:17.300 the people who are judging you are going to be like the GMs or the fans or that kind of thing.

01:38:22.980 And if they're judging you, they're going to be judging you for the unusual decisions.

01:38:27.940 So I think actually, if we go to Belichick, we can kind of see, if we go back to this idea of this

01:38:34.680 feeling of transparency, people are forgiving when there's a feeling of transparency. So if you're

01:38:40.240 Pete Carroll, at that point in his career, people are much more likely, and remember, even with him,

01:38:47.260 they really bashed him, but they're much more likely to sort of give you the benefit of the doubt.

01:38:51.280 So I actually saw a great talk by Toby Moskowitz. He's a really interesting guy.

01:38:56.200 There's a lot of stuff in sports analytics and quant and this kind of thing. And he was talking

01:39:01.820 about Belichick. And he said, when you look at his decision making, when he was with the Browns,

01:39:05.920 it was dismal compared to the analytics. And it's not actually until he wins two Super Bowls

01:39:11.640 that you start to see this decision making that's really starting to align with the analytics.

01:39:16.860 It's really starting to be like, I don't care if nobody's going to understand this decision.

01:39:22.560 I'm Bill Belichick, and that's the decision I'm going to make. But it's after two Super Bowls that

01:39:26.900 he gets there. Why? Because the fans are now tolerant of it. They're willing to tolerate the

01:39:32.180 stuff that they don't understand. Why did you go out for it on fourth and five there? What would

01:39:36.020 that's insane? No, well, because you're Belichick. So what we want to think about is,

01:39:41.020 when we think about these, what you said, the morbidity conference, what is that encouraging

01:39:46.940 in the people that you're trying to lead? You're encouraging the behavior that they naturally bend

01:39:53.960 toward, which is, I want to stay out of this lower right quadrant. So I don't want anything bad to

01:40:01.680 happen to me. But if it does happen to me, I'm going to make sure there's so much CYA there that

01:40:07.740 it's going to be okay, that nobody's going to sit there and say, I made a crazy decision. And you're

01:40:12.600 doing that because you're not, you're not exploring both directions, but particularly because you're

01:40:17.500 not getting in the room when something unexpectedly good happens. And here's the issue. When something

01:40:22.600 unexpectedly good happens, there's so much to learn from that.

01:40:26.300 Is that different, by the way, than when something good happens that shouldn't have happened based on

01:40:31.980 what people didn't know? Now I'm getting sort of Rumsfeld-y in a bit, but okay, I'm making this up.

01:40:37.280 So this could be complete nonsense. So I'm hoping you'll be able to come up with a real example.

01:40:40.560 Remember the whole Y2K thing? Everybody thought, oh my God, this could be a free for all, blah, blah, 1.00

01:40:46.520 blah. It turned out to be the biggest nothing burger in the history of civilization. Like I have

01:40:49.660 friends in college whose entire job after college was getting ready for Y2K. So nothing happened. 0.60

01:40:55.560 Fine. Now there's two ways nothing happened. One is nothing happened because there was really

01:40:59.780 nothing going on. And we just didn't understand that. And we did a whole bunch of stuff to be

01:41:04.040 prepared for it. And in the end, it didn't matter. There's another thing, which is, oh no,

01:41:07.760 it was a real problem. And all of that work that was done to make sure airplanes didn't fall out of

01:41:12.200 the sky, it worked. Those are kind of different. And I'm using the problem there is there's work

01:41:16.780 involved versus decision being made. So maybe this isn't a great example. I guess what I'm trying to

01:41:20.380 get at is you can have great outcomes because there was going to be a great outcome regardless of

01:41:26.920 your decision. And then you can have great outcomes where you actually made the wrong decision

01:41:31.660 and you got the right outcome. And then there's, you made the right decision and the right outcome.

01:41:35.680 I'm almost wondering, like, are there other historical examples in wars? I'm trying to

01:41:39.200 think of like D-Day. Was that invasion on that day the right thing to do? I mean, history will

01:41:44.200 forever be changed as a result of it in the right direction. But has anyone ever gone back and said,

01:41:48.940 well, you know what, based on the weather pattern, like Eisenhower got lucky. That shouldn't have

01:41:53.580 worked that day. And that was the wrong decision. They should have waited a day or something to

01:41:56.920 that effect. Yes. So number one, with Y2K, there's a fourth thing that you can say. So I agree 1.00

01:42:02.740 with all three things that you pointed out. It could have been good decision, good outcome. It

01:42:07.980 could have been, it would have happened no matter what we did. And so we were irrelevant. It could be

01:42:13.600 that we had a good outcome and it was all sort of a waste of time. So it was a bad decision to do

01:42:17.400 anything. It could be, it doesn't really matter because when you sort of look at what the range of

01:42:21.940 possibilities were given the unknown information, the downside was so bad that you were actually

01:42:27.720 supposed to protect against that regardless. And where are you going to waste a whole bunch of human

01:42:32.160 productivity doing it? Well, it's not really a waste because it just acts as a hedge against just in

01:42:36.660 case it's the worst case scenario. And in particular with something like that, we would want to protect

01:42:40.440 ourselves against that. But unless you dig into the win, you don't find that stuff out. And I think

01:42:46.480 that that's where the problem is. So with the Eisenhower example, what you said, it's such a

01:42:52.220 great model for how do we think about our own outcomes in our own lives. So this is what tends

01:42:57.480 to happen to us. And this is why I was saying, oh, it's actually not painful at all. When you have a

01:43:02.880 bad outcome to go in and look, it's what we naturally do. Something horrible happens to me. I'm looking in

01:43:09.440 there. I'm trying to find all the luck in there. I'm exploring what were my other options? Was there

01:43:14.200 something else that could have worked out better? How often was that decision going to work out

01:43:18.820 poorly? And I'm so excited. I'm so excited to go in and discover that I didn't do anything wrong.

01:43:24.700 I'm looking for that when it's a bad outcome. When it's a good outcome. When we land on the beaches

01:43:30.860 of Normandy, and it's a pivot point in the war, and we win, people aren't looking as hard to try to

01:43:38.780 figure out to really explore the counterfactuals. Well, what if I had done this? What if I had done this?

01:43:43.720 What if it had turned out this way? So one of the first steps to being a really good decision

01:43:50.540 maker, because we have this asymmetry in how willing we are to explore these outcomes. And

01:43:55.060 it's what you just said. It's we're all living our lives having a morbidity conference. Now,

01:44:00.460 the morbidity conference that we're having in our head is specifically to go find the luck in the

01:44:04.600 whole thing. But that's the only conference we're having. We're never having the my life is great

01:44:10.340 conference. Let me go figure out if that's because of my own decision making or because of luck or if

01:44:15.220 there was a better way or I could have actually opened up a whole other set of outcomes like

01:44:18.700 nobody's nobody's doing that. That is such an interesting point. You are so correct in that.

01:44:24.020 And as you're telling that story, I'm thinking of a sort of silly example in my life. But it's one that

01:44:29.860 anyone who knows me knows I take very seriously, which is how much I love archery. And I shoot a lot.

01:44:35.220 So when I am not in New York, obviously, when I'm back in San Diego, I'm shooting constantly. And I'm

01:44:40.320 always trying to push the limit of my capabilities, which means you're losing arrows. You're just

01:44:46.320 invariably if you're 60-70 yards away trying to hit something the size of a softball, you're going to

01:44:51.300 miss every once in a while. I'm generally losing an arrow to the tune of one a day. Now, these arrows

01:44:56.300 are very expensive. So this is becoming quite a problem. Every one I've missed, I can tell you a hundred

01:45:02.580 reasons why, and not bad luck. I can tell you the mistake I've made. I don't think I've once

01:45:09.340 ever given thought to the perfect shots. I don't think I've ever spent time thinking about that

01:45:16.780 ever. I think 90% of my energy goes into thinking about the 10% of the shots I didn't like. And I

01:45:23.940 thought that's the right way to do it. Isn't that making me a better shooter? What I'm hearing you say

01:45:27.800 is that's part of making you a better shooter. It's also going to encourage, I mean, if you think

01:45:32.180 about sort of societally, if it's the bad outcomes that are triggering the dives, first of all,

01:45:36.840 you're losing half of the opportunities to learn. If not more than half, because you might even be

01:45:40.960 making more positive outcomes than less. Right, right. Well, let's assume that the chances of having

01:45:47.160 an unexpectedly good outcome are symmetric. And let's make it that the thing that triggers

01:45:51.280 us doing a dive is that it's unexpected. And you can decide if it's like a one standard deviation away

01:45:59.000 or the standard deviation and a half or whatever it is, we can decide and that would obviously depend

01:46:03.500 on the uncertainty and how much ball there is and so on and so forth. But we'd figure that out. And

01:46:07.500 let's assume it's symmetric. So by only digging into the downside, we're doing two things. One is we're

01:46:12.360 losing half of our opportunities to learn. And the second thing is that we're reinforcing the risk

01:46:17.780 aversion. We're saying, hey, what we really care about is downside outcomes. So you better avoid

01:46:22.840 those. And three is we're completely quashing anybody ever doing anything unexpected, where

01:46:28.700 because if you do something unexpected, the chances that somebody is going to allow for luck or that you

01:46:33.180 yourself are going to allow for luck is the explanation is just going to kind of go poof. And

01:46:37.480 we don't like that. So there's all sorts of bad stuff that comes out of it. So when we get an outcome,

01:46:42.380 we want to basically do what you were just doing with the beaches of Normandy, which is to say,

01:46:47.960 here's the outcome. Before I make any conclusions about it, let me think about what are the other

01:46:54.800 possible things that could have occurred. So I have to think about what are the things that didn't

01:46:59.340 happen but could have those counterfactuals. And then I need to think about what was my preference

01:47:05.620 for those things? How much did I like each of those things? And how often were those things going to

01:47:09.880 happen? So it's just basically thinking about what was the payoff for those things. Obviously,

01:47:13.980 you can compare those to cost, but let's just simplify it. What were the possibilities?

01:47:18.720 What were my preferences for those possibilities? And what was the probability of each of those

01:47:23.820 possibilities occurring? And then, so that's the first step with the beaches of Normandy,

01:47:29.220 it ended up being a particular way. But given that they decided to do what they did on that particular

01:47:34.400 day, what are the other ways that it could have turned out with what probability would have that

01:47:38.840 have turned out and how much would we have liked or those things depending on what our goal was,

01:47:43.040 which in this case was to win the war. And then now we can take that and we can iterate it and say,

01:47:49.140 what if they had waited two days? What if they had done it two days before? What if they had done it

01:47:55.680 in a different way? They had gone from land instead of sea. Now we can start thinking about if they had

01:48:00.520 chosen other options, then how would that have compared in terms of what the expected value was

01:48:07.180 for the option that they actually chose? And you can do this for anything. So you can do this for

01:48:11.540 the archery. I assume that a bullseye is pretty unexpected. At certain ranges, it's more surprising

01:48:16.040 than it is not. Right. What you can say is you could actually go in and start forecasting it. I'm

01:48:20.140 going in at a particular range, given what I think that my skill level is, what percentage of the shots

01:48:25.680 do I think I'm going to get a bullseye? So forecast it. Say this is the percentage of times that I think

01:48:29.700 I'm going to get a bullseye. Now, if it's a particular day and you're well below that,

01:48:34.180 you would say, hmm, I wonder what happened today. What was the situation today where I was well below

01:48:39.480 what my expectation was for the number of bullseyes? But now if you're way above, you do the exact same

01:48:46.540 thing. And this is what we don't do. I can guarantee you, because I know this is the way that people

01:48:51.040 think. If you think, oh, I don't know how many times you shoot at the target. I've done so much of the

01:48:56.120 work you're describing without the important piece. But at 50 yards, for example, I know just

01:49:01.520 based on what I do, that I should be able to put eight out of 10 inside of a certain diameter. So

01:49:08.640 I spend a lot of time thinking about when I'm hitting six out of 10 inside of that range,

01:49:13.940 what's going on. But there's days I'm hitting 10 out of 10. This is what you do. You walk off and

01:49:17.980 you dance. You're like, look at me. Yeah, I don't think about why. You're like, whoa, check me out.

01:49:22.360 I'm great. And the thing is that, of course, you may have stepped up in skill. But wouldn't you like

01:49:28.900 to know exactly what the adjustment? What was it that actually clicked for you that day? What is it

01:49:33.460 that changed? And I know it's not that I stepped up in skill because it can vanish the next day. So

01:49:37.200 it's clearly something that is transient. Well, one of the reasons why it may be transient is because I

01:49:42.540 haven't identified it and marked it and incorporated it. Right, because it could be totally transient.

01:49:47.020 It could be that you get to a certain skill level and this is going to be your range. It's going to be

01:49:51.140 six to 10. Oh, well, by transient, I just mean what I'm really saying is I'm sure that I did

01:49:55.980 something better on that day. But it's transient because I didn't identify it and make it a part of

01:50:01.020 my permanent. It could be that if we could look at me under the six out of 10 day versus the 10 out of

01:50:06.000 10 day. I mean, I'm positive I'm doing something different. There is no way equipment, wind or any

01:50:12.400 other factor other than me explains that. But by only comparing eight to six, not six to 10,

01:50:19.080 I'm missing a greater disparity, which in theory should call out the more obvious deltas. It's not

01:50:25.800 just that you're missing half the time. It's you're using half the discrimination. It's like it's easier

01:50:30.420 to see that there are two lines if they're a foot apart versus a millimeter apart. So if I had to say

01:50:36.300 like if someone asked me what's the single biggest factor between a player who doesn't make a ton of

01:50:44.620 progress or make slower progress compared to a player who really, really makes a lot of progress

01:50:49.880 and becomes elite in poker, I would say that's it. That when you listen to someone like Eric Seidel

01:50:56.620 talk about poker, you would hardly know whether they won or lost the hand. All they're doing is

01:51:03.180 exploring what are the counterfactuals? Like what are the other lines that I could have chosen? Did I have

01:51:08.240 the person's hand right? What if I had bet a little bit more? What if I had bet a little bit less?

01:51:12.900 What if I hadn't played the hand at all? What would have happened if I played this hand that I didn't

01:51:17.260 play? And you would hardly know. Like if you listen to them, you would have no idea. In fact, probably you

01:51:21.740 would assume they had lost. Right, because it's so foreign to hear somebody. Because they're exploring so

01:51:26.460 much. You'd be like, whoa, you must have really lost that tournament. And they'd be like, no, I won the

01:51:30.820 tournament. And you move on. And what you hear from players who aren't making that kind of progress is it

01:51:37.060 goes two ways. When they talk about the hands they won, it's just sort of like I was great. And they don't

01:51:41.680 really spend a lot of time on it. When they talk about the hands they lost, it's a lot of exploration

01:51:46.120 of the luck elements. And the thing about poker, because the luck element in a given instance,

01:51:51.320 like in the short run, is so prominent that it's so easy to focus on the bad card that came.

01:51:59.920 So you'll have conversations with people where they'll talk about like, oh, I had this hand.

01:52:05.400 hand. And this person, it's called sucking out, they sucked out on me, meaning that they 0.86

01:52:10.400 happened to hit a good card. And to somebody who's thinking who spent a lot of time around 0.96

01:52:15.380 someone like Eric Seidel, you're looking at it and saying, well, you should have won that

01:52:19.500 hand way beforehand. Or maybe you shouldn't have played that hand at all. So for example,

01:52:23.480 like if I have a particular hand where you end up hitting a really good card, it may be that I

01:52:28.340 should have made you fold before then. Now sometimes it's you were going to be sticky, you were going to

01:52:33.040 stay around, I had mathematically way the best of it. And whatever, bad luck happened. But sometimes

01:52:39.560 it's I should have made you fold a lot earlier, I should have made you pay more for the privilege

01:52:44.820 of hitting that card, that actually the way that I bet the hand meant that you were making money to

01:52:50.700 my bet when I could have made you lose money to my bet, regardless of whether you were going to beat

01:52:54.420 me on that particular hand in the long run, I'd like to make you make losing decisions. Maybe I

01:52:59.300 shouldn't have played the hand at all. There's all sorts of other things that you

01:53:02.980 can be exploring in those situations, and they're not getting explored. Instead, what's getting

01:53:06.860 focused on is I can't believe that person sucked out on me.

01:53:09.780 So I have this framework for learning things that require skill. And I see a parallel now that I've

01:53:15.020 never seen before. So my framework, and I use swimming as the analogy for this, because I learned

01:53:19.540 to swim as an adult. So this was really the first time I explored this framework. But I do think it

01:53:24.120 applies to many things, languages, you pick it, frankly. So the framework is we start out as

01:53:29.880 unconsciously incompetent. So you take a person like me at 31 who doesn't know how to swim, you put

01:53:36.820 them in the water. There's no ambiguity about how bad I am. But I don't know why. All I can tell you is

01:53:42.480 I can't swim. I can't actually tell you why I can't swim. So the first step of learning how to swim is

01:53:48.300 becoming consciously incompetent. You actually have to understand why you're really bad at this. Oh, once you

01:53:55.040 start to understand Archimedes principle, that's like a very important part of understanding how

01:53:59.880 to swim. The more of you that's underwater, the more buoyancy there is to hold you up. Once you

01:54:05.280 start to understand how Bernoulli works, oh, all of a sudden, there's this difference about the

01:54:10.660 velocity of a fluid over a body, etc. And then by the way, you're no better as a swimmer in those

01:54:15.460 two boxes, right? The difference between... Right. I just want to know the percentage of people who are

01:54:19.700 learning swimming who cite Archimedes and Bernoulli. I just love that that's the way that you're...

01:54:26.460 I don't know if I still have them, but I would love it if I did. I used to journal so relentlessly

01:54:30.960 about swimming. I had reams of journals that I would write in every single day of my free body

01:54:39.100 diagrams of my body in different swimming positions and trying to figure out ways to make this

01:54:44.720 less bad. When I got into this, I became so obsessed that I swam twice a day every day.

01:54:50.760 This model makes so much sense to me. Well, you don't actually start swimming until you reach the

01:54:55.280 third box, which is now you are consciously competent. Now, the problem is you can't do that

01:55:00.960 for long. So you tend to vacillate a lot between the second and third box, which is consciously

01:55:06.980 competent, and then you go back to being consciously incompetent. But as long as you can maintain focus,

01:55:13.300 you will vacillate between those. And usually it's your physical fatigue that will push you

01:55:18.500 from the third box back to the second box. Now, when you become a really good swimmer,

01:55:23.860 like Michael Phelps, you transcend to this fourth box. You are now unconsciously competent.

01:55:30.060 Everything becomes sort of autonomic at that level. Now, thinking about this through decisions,

01:55:34.660 as you were telling the story about Eric, I'm thinking about the first level. So let's just call it

01:55:39.100 four levels. Now, the first level is you only examine losses through the lens of luck. So

01:55:45.900 that's our default. I would argue that takes no effort, just as being unconsciously incompetent

01:55:51.580 is our default. Layer two or level two thinking is I only examine my losses through the lens of

01:55:59.980 luck and skill. That's great.

01:56:02.240 So you're taking some responsibility, but it's only triggered by the loss.

01:56:05.340 Only triggered by the loss. Then you have a level three, which is I'll do that, but I will also

01:56:10.840 now examine my wins through skill. And then the fourth level, which I'm guessing is exactly where

01:56:16.980 someone like Eric is, is no, I'm going to look at wins and losses through luck and skill.

01:56:23.560 Yeah. So what I would say is I think that the thing about examining wins, I do think that examining

01:56:32.300 wins through the lens of skill is pretty natural. We don't spend a lot of time on it. It's kind of

01:56:38.220 an assumption of skill. I would put wins down in level one. Let's put it this way. I would focus more

01:56:43.820 on, I would actually think about it, not just in terms of, are you examining the quadrants that are

01:56:52.060 discordant in a symmetric way? So what I would say is that on the level one thinking is I lean

01:57:00.940 on the discordance on the loss side and I lean on the concordance on the win side. And I just kind

01:57:07.260 of do that naturally. And I don't spend a lot of time thinking about either of them.

01:57:10.940 That makes sense.

01:57:11.980 I don't spend a lot of time thinking about either of them. Then I think that level two is you're still

01:57:18.300 leaning on concordance on the win side. You're not examining those very much, but you do start

01:57:22.380 to take the losses- At least you examine the discordant losses.

01:57:24.700 Right. You examine the concordant losses now. So you examine the losses that are actually due to

01:57:29.740 your own undoing. Then I'll get to the fourth level. Then I think that level three is I'm looking

01:57:39.820 in all four quadrants. So I'm looking at wins that are due to skill, that are due to my good decision

01:57:45.340 making. I'm looking at wins that are due to my bad decision making. I'm looking at losses that are

01:57:50.540 due to my bad decision making. I'm looking at losses that are due to my good decision making. And

01:57:55.100 I'm trying to, I'm really trying to sort of look at those all equally. Where I think the fourth level

01:58:00.140 comes in is when it's concordant on the win side, that you don't accept it. In other words, that you

01:58:09.900 say I made good decisions and I won, but it does not mean there wasn't a better way. So let me actually

01:58:17.900 start to really clean up in there and say, it's not about I made a decision. It was a winning decision.

01:58:25.580 I was positive expected value for it. And so I'm done. It's now I so care that I got onto what we

01:58:34.700 would call the primary line of play, that I really came up with the best possible solution that I could,

01:58:39.260 given what I knew, that I won't be satisfied with just finding out that there was concordance.

01:58:44.620 Like I want to go beyond that on the win side. I think what we're going to do in the show notes

01:58:48.140 is literally take everything you just said and turn it into a printable one pager that every one

01:58:54.060 of us should carry with us. Because that's, that is a very elegant way to think about that. I promise

01:59:00.620 you I've never done what you just said. Like I've never once had a level four thought in my life.

01:59:05.580 Well, first of all, let me just say that I only thought about it because you gave me the levels,

01:59:10.300 but it's that last bit. So right now I'm writing a workbook, very close to finished with it,

01:59:16.060 where in chapter three, I really dig into that piece, which is when you have a good outcome,

01:59:23.580 and it turns out it's from good decisions. What are you doing with that? Are you done? Or are you

01:59:28.860 trying to figure out if there was even better one beyond that? And here's the reason why we don't like to

01:59:33.660 do that. Because when we think about what sort of creates the fabric of our identity,

01:59:40.700 our beliefs are the threads that our identity is woven out of.

01:59:43.820 By doing a level four thought, you risk, they're undermining that.

01:59:46.460 Tearing a hole, tearing a hole in it. Because one of the things that we want to do is feel like that

01:59:51.820 fabric is strong. Like our beliefs are good and we're competent. I mean, this idea of competence and

01:59:58.300 being able to bank credit for the way that things have turned out in our life is so core

02:00:05.100 to what forms our identity that when you don't stop at, my decisions were good and I had a good

02:00:12.540 outcome. When you actually dig around in there, you're risking now turning something that was a win

02:00:18.780 into a loss. So this is where I love your idea of the levels. So if we think about that level one,

02:00:25.340 like I won, it must be good. Why wouldn't you go further than that and say, well,

02:00:28.860 maybe I won because of bad luck. You're turning a win into a loss. And in this particular case,

02:00:32.860 we're taking win as the outcome. I had a good outcome. And you sort of want to just stop there

02:00:37.820 because you don't want to dig down and find out you might have made a bad decision because now you

02:00:42.140 turned a win into a loss. So that's where when you get to level two, you're willing to do that.

02:00:46.940 You're willing to say, I want a long run win. And what that means is I might have to turn this

02:00:51.900 outcome that was a win into a loss. When you get to level four, what you say is,

02:00:55.980 okay, I had a good outcome. I looked and I saw that my decision-making was pretty good,

02:01:01.240 but now I'm willing to turn that win into a loss in order to get long run better.

02:01:05.760 Yeah. You were basically completely exposed at that point. I can imagine how your book became

02:01:11.260 immediate fodder for every hedge fund manager and so forth. Because again, I think the advantage

02:01:16.440 of that type of investing bears some resemblance to poker, which is you can't really hide your

02:01:23.220 decisions. Your outcomes are very clear. And which is not to say there's not a huge element of luck,

02:01:28.800 just as there is with poker, but it's basically an industry where your decisions are exposed.

02:01:34.220 Every decision you make in the end, you're going to be able to trace it to an outcome. And frankly,

02:01:40.280 unless you're not in the business of making money, you have to be wed to good outcomes.

02:01:44.260 So the poker player wants to do well, the hedge fund manager wants to do well. There's nowhere

02:01:48.760 to hide, I guess, is what I'm trying to get at.

02:01:50.440 I agree in theory that there's nowhere to hide. The interesting thing that I think is how long

02:01:57.880 people can hide from it. That's what I find so fascinating is you look at poker and you're like,

02:02:04.640 you're getting answers right away, right? Like you win or lose. So here's the problem. It's like,

02:02:09.280 okay, at the end of the year, somebody lost, right? But why? How far are you willing to go

02:02:15.980 to say, I just got bad luck? That's the question. And I'm just really fascinated that people can go

02:02:23.440 really far. They can really convince themselves that they just got bad luck. And I think that that's

02:02:29.760 true. Obviously, if you're running a hedge fund, you have investors that you're accountable to.

02:02:34.780 But we all know of hedge fund managers where they had a couple of good years and then people are

02:02:40.480 sticking with them. It's like you have a couple of good years to start. There's this path dependence

02:02:44.200 to this. And it's true of poker. Like if a poker player comes out of the gate in the first six

02:02:47.720 months and loses everything in sight, they're very unlikely if they continue to lose. They're going

02:02:52.660 to say, oh, I must be really bad at this. But for every poker player who comes out of the gate

02:02:56.640 with a terrible first six months, there's a poker player. I actually remember way back when,

02:03:02.040 a long time ago, there was a new player who came on the scene. And they won a World Series

02:03:08.880 of Poker Bracelet. I mean, they were really young. I mean, early 20s. And nobody had ever heard of

02:03:13.120 them before. And they won. And I thought, oh, you know, that's nice. Because that happens, right?

02:03:16.880 Obviously. Then they won a second one. And I was like, oh, that might be just the ruin of them.

02:03:22.440 Now, why would I say that? Like they had won a second World Series of Poker Bracelet. Because

02:03:25.900 I understood this path dependence, right? That if you come out of the gate, and you do really well,

02:03:32.040 your ability to fool yourself afterwards for a very long time, it becomes almost infinite.

02:03:37.760 And you see that in the hedge fund world all the time. Obviously, if an investor comes along,

02:03:41.920 and they don't show any results, by the way, sometimes unfairly, it becomes very hard for

02:03:46.540 them to recover from that. Sometimes unfairly, because maybe it just went against them for that

02:03:51.240 year or whatever. But when it's the reverse, when they come out of the gate strong, you can fool

02:03:56.640 yourself for a really, really, really, really long time. And that's the thing that I find so

02:04:00.800 fascinating. That's what really kind of lights me up. And trying to figure out the capacity

02:04:06.380 for self-deception is so a bottomless well.

02:04:11.180 Is there an industry that you look at where you just see, on average, it is more amenable to

02:04:17.800 higher level thinking the way we just described the four levels in poker, like this framework?

02:04:23.740 Is there a place where either through some externality of the way the industry is set up

02:04:29.200 and the rate of time being short between decision and outcome and some sort of transparency, that

02:04:35.320 we have a space where there's a greater incentive to rise to that level?

02:04:39.380 First of all, let me just say this, that obviously, we're only talking about places where in the

02:04:44.500 shorter run, luck is going to have a strong influence. So when you look at sports that are

02:04:50.980 really highly under the influence of skill, people are getting selected out. Nobody's

02:04:56.120 fooling themselves. Well, actually, it was just bad luck that I didn't get to be a professional

02:05:01.120 NFL linebacker. I mean, it might be if you got an injury, but I'm 5'5 and I weigh 130. Such

02:05:08.580 bad luck that I wasn't. People are just like, no, I just wasn't good enough. So we know that

02:05:12.920 in cases where it's very transparent that if you're not doing well, we can trace it back to

02:05:18.920 skill. So the more chess-like we are, the less the capacity for us to fool ourselves.

02:05:24.280 If I lose to you in chess, it's very hard for me to say that I got bad luck because I don't

02:05:28.580 know, where's the luck? I don't know, go find it for me. There's a little bit of luck, but

02:05:31.820 not much. So the more on the chess range of things we are, the less the capacity to do

02:05:37.640 that. So first of all, we've got to be in an environment where there's a strong short-run

02:05:41.160 influence of luck. Now, the tighter the feedback loop, the more feedback you're getting,

02:05:48.040 the more that somebody needs to rise to the occasion. So if you were to look at what's

02:05:52.760 the capacity, which individuals are going to thrive more in those systems, definitely

02:05:57.580 the ones who are much more willing to dig in and dig around. So people who are in higher

02:06:03.420 frequency trading, for example, where those feedback loops of the time horizons are shorter,

02:06:07.600 I think are going to be more likely to be thinking in this way. And if they're investing their

02:06:13.240 own money.

02:06:14.680 Ah, that's a very interesting point. It's got to, Nassim, right? You got to have skin

02:06:18.100 in the game. Are there poker players, by the way, that play with other people's money? And

02:06:21.160 by that, I mean not money that they won from somebody else, but I see. Okay.

02:06:25.320 Yeah. So there's backing situations. Yeah. There's the same thing. The less you're trading

02:06:29.220 on sort of reputation and the more that you're trading on your own, your own dollars. Also,

02:06:33.360 I think that you're more likely to end up in a spot where you're thinking that way.

02:06:36.600 Basically, you have tons of skin in the game. There's a high element of luck and skill and

02:06:43.040 the feedback loop is short. Those are three situations that will tend to push you towards

02:06:48.440 either abject failure or getting better, moving up the ladder of this thinking.

02:06:53.880 It's a little chicken and egg. So I don't know. Is it the people who succeed are more naturally,

02:07:01.060 kind of more naturally tend to have that kind of thinking and obviously it gets reinforced or

02:07:06.280 what would Eric do if he wasn't the best poker player in the world?

02:07:10.400 Well, I know what he would do because he was an options trader.

02:07:13.480 Okay. So again, options trading, you could argue that it's sort of similar, right?

02:07:17.540 It's incredibly similar. It's almost the same.

02:07:19.880 But would he go and be a good CEO of a blue chip company? Like if he all of a sudden tomorrow,

02:07:25.840 assume he had the domain knowledge. Okay. So he could go in and be the CEO of General Electric

02:07:30.620 or Home Depot or Exxon Mobil or something where you could put a chip in his head that gives

02:07:35.720 him the domain expertise to understand exploration and production or whatever. But could you move

02:07:43.360 that industry or that company within its industry relative to its peers by bringing that layer of

02:07:49.080 thinking?

02:07:49.900 Yes. So that's what, that's, that's kind of what I want to say. Is there a little bit of a chicken

02:07:53.580 and an egg issue? Because I have met people across industries where there are people who are thinking

02:07:59.020 this way, where there are people who are really hungry for it. Now, in terms of this idea of like,

02:08:03.460 what are you doing when you find out it was pretty good decisions and good outcomes? Or are you only

02:08:08.340 doing postmortems and not thinking about it in terms of expectancy? That's a problem that you see

02:08:13.300 across all industries. But also across industries, you see people who are really hungry to try to

02:08:18.840 incorporate that into their solution space. How am I dealing with this? How am I running my teams?

02:08:24.600 So if you just take the investing world, I see people who think that way all the way from people

02:08:30.940 who are trading options to people who are doing seed round venture capital, much different than

02:08:37.480 late stage venture capital, where you might, the time horizon now is faster. I mean, not the same

02:08:42.740 speed as options. But if you're like in the seed round, we're talking horizons, it could be a decade.

02:08:47.800 There are fund managers who are in long short funds, where they might put on seven positions

02:08:53.100 in a year, you're hardly getting any data. And they're, they're really looking to think this way.

02:08:58.040 Because what's interesting is that while trading options might foster this kind of thinking better,

02:09:05.760 because you're getting feedback more quickly, while you might end up getting sort of selected out by

02:09:12.040 the people who you're accountable to quicker, while maybe that particular type of mind would tend to get

02:09:17.720 drawn towards say options trading better. The issue is that in absence of the world telling you

02:09:24.700 quickly, this type of thinking actually becomes much more important. Yeah, it becomes more important

02:09:29.880 and more difficult. Right. Because when you spread the feedback loops out, now, the way that I always

02:09:35.960 think about it is uncertainty gives you the leeway to allow bias to come in. And that's the difference

02:09:43.600 between the chess and poker. In chess, because you've taken a lot of uncertainty away, you don't have

02:09:47.660 the leeway to fool yourself. That is an absolute gold point. But in poker, you do. So we can sort of

02:09:52.760 think about if you look at the difference between say trading options and something that has a much

02:09:56.420 longer time horizon, the longer the time horizon, the more capacity you have, the more leeway the world

02:10:03.060 is giving you to fool yourself. So then this kind of thinking, really thinking in this systematic way,

02:10:09.760 really modeling the world in this way becomes much, much, much more important. It becomes a bigger

02:10:15.080 imperative for you to be poking, for you to be thinking counterfactually, for you to have people

02:10:19.540 challenging your ideas, for you to be trying to think about, am I doing this because it's status

02:10:26.940 quo? And if it doesn't go well, I won't get blamed. Am I innovating enough? Am I allowing people to

02:10:33.680 challenge my ideas? Am I challenging my ideas myself enough? Am I looking at that discordant box that has

02:10:41.860 to do with good outcomes? Or am I only willing to look at the discord when it's a bad outcome?

02:10:47.500 When the bad outcome is concordant, am I like super excited about that? Or am I imagining, well,

02:10:52.780 maybe there was a better way? How am I digging into all of those spots? And then if you're in a

02:10:58.020 leadership role, how am I propagating? Or am I not propagating these problems across the team?

02:11:04.240 Am I telling the team through my actions that they should be terrified of bad results? As I'm thinking

02:11:10.000 about the way that the world is giving me the leeway to be able to sort of have my beliefs lead me

02:11:15.140 around on a leash, am I infecting my team with that same problem? Because now I can be creating this

02:11:22.020 whole other problem. And it's such a long time horizon that maybe the world's going to tell me

02:11:26.520 that this was actually problematic. You should be digging into that as if you're getting the answer

02:11:31.440 tomorrow, as if you're playing chess, and you're going to find out right away. And you should be doing

02:11:36.520 everything you can to try to make sure that you don't find out you hung your rook.

02:11:41.740 This to me is why swimming is harder than riding a bike. You can teach anybody to ride a bike

02:11:45.560 relatively quickly for a couple of reasons. One, the feedback loop is very quick. Like anytime you're

02:11:51.180 out of balance, I mean, we just naturally internally, our inner ears allow us to sense

02:11:55.840 that disequilibrium instantaneously. So you get the fastest feedback loop imaginable,

02:12:02.160 and the consequences are quite high. It hurts significantly when you're not in balance.

02:12:06.440 Contrary to popular belief, swimming is about balance. It's the exact same thing. You just

02:12:10.160 happen to be in a medium that is so much more dense than air that the feedback loop is much slower,

02:12:16.300 and the consequences are much more gentle. It doesn't hurt when you're out of balance. In fact,

02:12:21.980 if you fell every time on a bicycle that you were out of balance while learning to swim,

02:12:26.100 you'd never ride a bike again. You'd be so black and blue. So in that sense, swimming is

02:12:31.600 gentler, meaning you have more excuses. It's longer time horizon, but it's that much more

02:12:37.920 important that you do the examination. So if anybody can mindlessly ride a bike very soon

02:12:43.120 after being shown how to do it, you can't mindlessly swim for years after learning how to swim. It's that

02:12:49.640 hard. Swimming then becomes the example of where it's more important that you bring that level of

02:12:56.140 thinking if you want to make any progress. I agree. And let me add something to this,

02:13:01.720 which, correct me if I'm wrong, but I think this is in the same space as you're thinking.

02:13:05.920 Sort of thinking about this soft landing issue, the wider the skill gap between you and the people

02:13:15.060 that you're competing against, there's so much more cushion for a soft landing. So this is something

02:13:22.060 that I sort of think about is by not exploring that, sure, I made winning decisions and I won by

02:13:29.760 not willing to go beyond that. Imagine if you're in a situation where, let's say we're in some sort

02:13:35.340 of zero sum game. So we're trading against each other. And if I were making perfect decisions against

02:13:42.320 you, every time that we put a dollar on the table, I would get a quarter. But because of the decisions that

02:13:49.940 I'm making, every time we put a dollar on the table, I'm getting a nickel. So if we think about

02:13:54.760 those quadrants, right, when I look at I have a good outcome, I won a nickel. And my decision making

02:13:59.260 was good because I was positive expectancy, right? I was winning a nickel. I was winning 5%. But by not

02:14:06.280 exploring, I don't see the 20 cents that's sitting there that I could get every single time. And that's

02:14:12.820 the problem with a soft landing. That's the problem with having that cushion is it doesn't let you know

02:14:18.840 that there's something better that you could be doing.

02:14:22.100 I know we got to wrap because we both have to get moving. But there's one topic that you very

02:14:26.260 loosely, I think we were going to go down the path of it by name, but then we didn't. But I want to

02:14:31.740 bring it up because it's such an important concept in my world. And it's something I have done for such

02:14:38.180 a long time, but never had a great name for it. And when I read your book, I was like, oh, awesome. I'm

02:14:43.680 going to be plagiarizing this all day long. And it's the idea of backcasting.

02:14:48.100 Oh, yeah.

02:14:48.660 A moment ago, you used the term forecasting.

02:14:51.020 Right.

02:14:51.320 We're all about forecasting. Well, I want to be doing this in Q4. And therefore, I need to do this

02:14:57.120 today and this tomorrow and this tomorrow. And I'll tell you how I use it. And then I'll...

02:15:01.780 And let me just say for the record, backcasting is not a word that I coined.

02:15:05.640 Oh, all right.

02:15:06.480 I don't want to take credit for that.

02:15:07.660 I got to reference you who references someone else.

02:15:08.860 You can say you read about it in my book. I didn't invent that word.

02:15:11.720 All right. Thank you for clarifying that. So prior to reading your book, I would just refer

02:15:16.120 to it as reverse engineering, but that never really had any resonance with people. But now

02:15:19.960 when I say backcasting, it forces me to explain the contrast between backcasting and forecasting,

02:15:24.780 whereas reverse engineering doesn't have a striking of a contrast. So the place that I use it

02:15:30.400 is in understanding the exoskeleton of the body. So the actual physical structural demise of ourselves.

02:15:38.220 So we're all sitting here thinking about exercise and everybody knows they should exercise,

02:15:41.860 but nobody really can give you a great reason for why. We've been sitting here talking about

02:15:45.520 the outliers, professional athletes, but outside of people who get paid to do something with their

02:15:50.760 body, what are the rest of us doing? It's amazing how often you push people on this. They can't give

02:15:55.140 you a great answer. And because I ask everybody under the sun, this question, I'll tell you all answers

02:15:59.640 basically fall into these three buckets. So if you're asking someone who's not getting paid to do a

02:16:04.460 sport, why do you exercise? 90% of the answers are, I like the way it makes me look. I like the way it

02:16:10.940 makes me feel. And I like the freedom and flexibility it gives me with what I eat. Some combination of

02:16:17.900 those three. There are other answers that show up a lot less frequently, such as I love the social

02:16:21.460 interaction that comes with it and things like that. Now I would argue all of those are fine reasons to

02:16:25.880 exercise, but I don't think they're good enough. The good enough one only comes when you start

02:16:29.860 thinking about backcasting, which is what do you want your body to function like the day before

02:16:36.260 you take your last breath? Really think about this. So I've taken this to an extreme and people have

02:16:41.440 heard me talk about this in the podcast before, but I come up with this idea of the centenarian

02:16:44.540 Olympics, which is instead of training for like an Olympics that's now where you would actually have

02:16:50.820 to pull vault and run a hundred meters in under 10 seconds, think about what an Olympic games would

02:16:55.920 look like of a hundred year olds and ask yourself, what do you want to be able to do in that Olympic

02:17:01.280 game? So I have like 18 things I want to be able to do when I'm a hundred, if I get to be a hundred

02:17:05.400 and they're all quite mundane, but they're all very pragmatic and practical. One of them is being able

02:17:11.360 to do a 30 pound goblet squat, which again, I could do today blindfolded on one leg a hundred thousand

02:17:17.120 times, but at a hundred, that's actually going to be quite a task. And it's important because that

02:17:21.960 replicates picking up a toddler, which presumably if I'm a hundred, I've got great grandkids. I want

02:17:26.780 to be able to pick them up. I want to be able to stand up off the floor using no support and only a

02:17:32.780 single point of my own, meaning one arm and both my legs and get up off the floor without help. Can I

02:17:38.640 do that today? Of course. Can you do that today with your eyes shut? Very few hundred year olds can do

02:17:43.960 that. So I've kind of 18 of these things I want to be able to do. And that is backcasting. It's going to

02:17:50.620 the finish line and saying, if I can do that at a hundred, what was I able to do at 90? If I can do

02:17:56.580 that at 90, what must've been true at 60? And before you know it, I realized, wow, if I want to do that

02:18:02.740 at a hundred, now that I'm 46, I actually have to be able to do all of these other things. So there's

02:18:06.740 this whole matrix that comes out of it. That's why that concept for me just resonated so much. Cause I

02:18:11.720 really, I don't think medicine spends enough time backcasting. Yeah. So let me try to add another

02:18:16.600 benefit of backcasting. So just to be clear, backcasting is, so when we forecast, what we think

02:18:22.100 is there's a goal that I would like to reach. How do I get there? And in generally what you're thinking

02:18:26.920 or what are the things that I'm doing in order to get to this goal? When you backcast, what you say

02:18:31.200 is it's the day afterwards. So I have some goal I'd like to be able to do X in a year. And now it's a

02:18:38.000 year and a day and I achieve my goal. And what you ask now is you look back and you say, how did I

02:18:44.860 actually achieve this? And you can also do something called a pre-mortem. So a pre-mortem

02:18:49.760 would be, I have this goal that I'd like to achieve. I'd like to achieve it in a year. It's

02:18:54.340 now a year and a day and I failed. Why did I fail? Why did I fail? Now here's an added benefit that I

02:19:00.080 would like to offer to you. When we forecast, when we think about, here's my goal, how do I get there?

02:19:06.440 We're thinking about the things we're doing. Here are the decisions that I make along the way.

02:19:10.960 So we're thinking about all the things that are within our decision-making power, within our

02:19:15.460 control that are in the sort of skill side of the equation. But what happens when you backcast

02:19:20.440 and you say, I actually succeeded, how did I get here? But particularly what happens when you do a

02:19:26.240 pre-mortem and you say, I failed, how did I get here? Now you allow luck into the equation. So as an

02:19:34.120 example, if I'm thinking about when I'm a hundred, I would like to be able to do a 30 pound goblet squat,

02:19:41.820 which I've never heard before, but now I'll say that because anyway, I learned that, but I'm now

02:19:46.700 a hundred. It's the day I've turned a hundred and I cannot do that thing. Why? Well, I can think of

02:19:51.980 some things like I didn't actually exercise enough, but I can also think of, I tripped and I fell and I

02:19:57.720 broke my hip now tripped and fell. That can happen to anybody. Probably not a matter of skill. It just

02:20:04.340 something can happen. Like you could hurt yourself. But the break, my hip might have to do with my bone

02:20:08.660 density. That's going to cause you to explore other things that maybe wouldn't be as obvious

02:20:13.160 because you saw nobody's thinking you don't see that bad luck can occur if you're not working

02:20:20.160 backwards. And then to your point, once you work backwards, you can now ask yourself some

02:20:24.400 questions because you see the luck. You can say, okay, if it's on the good side, can I increase the

02:20:30.900 chances that that thing happens to me that I see is luck is going to intervene in my way.

02:20:35.980 Can I increase the chances that's going to happen for me? Okay. So you could ask that,

02:20:39.600 but then on the downside, which is really important to explore, let's say the breaking the hip thing,

02:20:44.200 can I decrease the chances that I break my hip? Yes. What are the things that I could do? If the answer

02:20:49.460 is no, then you can say, are there any hedges available to me? Can I hedge against this bad

02:20:56.040 luck thing happening to me? Sometimes the answer to that is also no. Regardless, what you can say is,

02:21:02.420 okay, I've got my hedges set up. I'm doing what I can to decrease the chances that it occurs, but

02:21:07.080 obviously it still could. So let me think now before that bad luck intervenes, what am I going to do in

02:21:13.640 response? Because if I plan for that in advance, I'm going to be making decisions right now in a

02:21:20.660 calmer state of mind where I'm not pants on fire, freaking out, rather than being reactive to what

02:21:26.940 the world might deliver me at the time. So if I've already thought about it, if I already have a plan

02:21:31.480 in place, I'm not wasting time. I'm going to be much more calm. I'm going to think about it. So this

02:21:36.520 is one of the big difference between working forwards and working backwards is that when we work forwards

02:21:40.340 and we're thinking about how do we execute on any kind of strategic plan, because we can think about,

02:21:44.120 I want to be able to do this thing when I'm a hundred and we're developing a strategic plan for

02:21:48.500 how we get there. We really focus on what are the things that we're going to do that are going to get

02:21:53.540 us there. And we don't say, what are the ways that luck might intervene? But when you work backwards,

02:21:59.520 you naturally start to see the luck. And then you can start to explore all of these questions that

02:22:04.140 then makes your strategic plan much better. It makes you explore different stuff.

02:22:08.220 Now, does your workbook cover exactly this type of decision tree that we just went through?

02:22:12.500 Yes.

02:22:12.780 When is this coming out?

02:22:13.660 A year from now. So I'm just finishing up the draft now. And then you have to lay it out. You

02:22:19.500 know how that goes with books. So basically what I'm doing with the workbook is I'm taking

02:22:23.320 these concepts that I talk about broadly and sort of bigger idea way and thinking in bets. And I'm

02:22:29.280 saying, okay, how do you practically make a decision? How do you build out a decision tree? How do you

02:22:35.060 think about the way that luck might intervene? How do you think about your own decision making?

02:22:39.820 How do you see the luck? How do you communicate with other people such that you're making sure

02:22:45.820 that you're getting the purest feedback from them, that you're not biasing their feedback,

02:22:51.280 that you're actually hungry for the information and you're seeking the feedback, which is, by the way,

02:22:55.400 the first step, which most of us don't do. Most of us aren't seeking the feedback. So when you're

02:22:59.940 hungry for the feedback, you're actually thinking about what does the future look like? What could the

02:23:04.820 outcomes be? How am I going to react to those things? How do I understand the sum of my experience

02:23:10.400 and what that's supposed to teach me? So it was really trying to think about how do I allow people

02:23:15.380 who don't do this for a living, who don't spend their time building out probabilistic decision

02:23:21.000 trees, and this is not the way that they look at the world. How do I take somebody like that and

02:23:25.660 show them the value and teach them how to actually think within this framework?

02:23:29.940 Well, I think that means we're going to have to sit down and talk in about a year.

02:23:32.300 Yeah, I hope so. I want to also just say one thing just for the listeners, because I would like you

02:23:37.280 to put it in the show notes. So I'm going to say it right here. A lot of this stuff that we've talked

02:23:41.140 about with luck and skill, I really want to recommend. Well, first of all, on the forecasting

02:23:44.880 stuff, let me just, again, recommend super forecasting, which I know is going to end up in

02:23:48.740 the show notes. But there's a really, really wonderful book by Michael Malbison that's called The

02:23:53.880 Success Equation. And it's all about the influence of luck and skill in our lives. It's got a lot of

02:23:58.980 sports stuff in it, but just a lot of really digging down and drilling down into understanding

02:24:04.480 luck and skill and how you sort of understand what the difference is between the range of

02:24:10.020 activities and sort of where you can see more influence of skill, where you can see more

02:24:13.860 influence of luck, how you can sort of parse those apart. This huge influence on our lives is luck.

02:24:20.220 And he really drills down into that in a way that if people are really interested in the

02:24:25.860 conversation that we had today, then I guarantee they would be really interested in the success

02:24:31.240 equation. And Malbison also, I mean, you can just search him and you can see a whole bunch of videos

02:24:36.100 of his where he talks about luck and skill. A lot of his writing talks about luck and skill and sort of

02:24:42.860 separating the two and understanding the different influences. And yeah, I just can't recommend that book

02:24:47.520 more. We'll put together a pretty robust set of notes here as we always do. And it will point to

02:24:52.080 everything we've spoken about. Plus, I'm sure after the fact over the next couple of weeks, you and I

02:24:56.720 will have back and you'll think of something else that you want to hear. So I'd love to make this the

02:25:00.320 sort of the compendium for all these things. And then, of course, I can't wait to sort of talk about

02:25:03.520 this workbook. It sounds like it's kind of an amazing tool. I mean, again, I would say like in the

02:25:07.940 last few hours, I've realized that I'm not nearly as good a decision maker as even I think I am. And I don't

02:25:13.420 say that to be sort of arrogant. I just think, look, I understand decisions at least as well as

02:25:18.680 anybody else has been my bias. And I'm at least as probabilistically savvy. But I realized like

02:25:24.120 there's tons of blind spots that I'm still experiencing and I'm not even close to approaching

02:25:29.660 the potential that that a person would have to make better decisions. So that said, I find myself

02:25:35.340 somewhat overwhelmed by how complicated this is. I'm already thinking like Travis is going to put

02:25:40.720 together some great tear outs that will hold us over until your book comes out. But having a book

02:25:45.880 that could be sort of a guidepost to a decision matrix would be fantastic. And let me just for

02:25:51.740 people who have listened to this and felt sort of overwhelmed. It's funny because this happens to be

02:25:56.200 the chapter that I'm writing right now is that it's really, really good to have this framework to be

02:26:02.260 thinking about decisions and to really recognize the probabilistic nature of the world and the way that

02:26:08.020 things turn out for you and understanding what the structure of a really good decision is. That being

02:26:12.900 said, that doesn't mean that you have to sit down with a slide rule for every decision that you make.

02:26:18.520 And in fact, for the majority of decisions that you make in your life, you can actually decide quite

02:26:23.320 fast. And what's interesting is that the way to really understand when can I decide fast is to

02:26:31.260 understand this broader framework, this broader framework that would allow you to take the bigger

02:26:35.280 decisions in your life and actually start to think about them in this way where you're really

02:26:38.660 forecasting them is actually the way to understand when can I go really fast. And what it will allow

02:26:45.160 you to do is stop taking 15 minutes to decide what to order in a restaurant, because you have to

02:26:51.120 understand this framework in order to sort of get when you can go fast and when you can go slow.

02:26:55.800 Because when I do talk to people about this kind of framework, they do feel a little bit

02:26:59.540 overwhelmed. And they're like, Oh my gosh, how am I ever going to make a decision again? Like I'm

02:27:04.080 going to need a supercomputer in order to do it. It's very few decisions that you actually

02:27:08.220 need to take a ton of time on. Mostly you can decide fast. And on the ones that are bigger

02:27:14.140 decisions, embracing the probabilistic nature actually gets you out of the analysis paralysis.

02:27:20.160 Because what you realize is once you've identified your options and you've got an option that you see

02:27:24.380 is better relative to the other ones, you stop focusing on achieving absolute certainty and start

02:27:30.720 focusing on achieving relative to the other things that I could do. This one definitely looks better

02:27:35.420 to me. And because I recognize that there's all sorts of stuff I don't know, and there's all sorts

02:27:40.700 of different ways the world can turn out. And as long as I'm taking the things that occur in my life

02:27:45.300 and trying to learn from them in a real way, I'm not going to be so terrified of sort of the

02:27:49.980 downside. And that really frees you up. You've said that so, so elegantly. And I think that

02:27:55.020 reiterates this sort of the notion that intuitively people get it right. You go fast on the straight

02:27:59.960 away and slow on the curves. This framework shows you where the curves are. Yeah. And that frees you

02:28:04.060 up to go faster in the straightaways. Yes. And interestingly enough, it frees you up relative

02:28:09.620 to what you were doing before to go faster on the curves as well. Absolutely. To not try to figure out

02:28:15.700 in that moment, if I don't know the exact right path to take, I'm paralyzed and I just need to slam the

02:28:22.120 brakes on, but to recognize when, well, this path is going to be good enough. I'll spare all the

02:28:27.080 listeners the race car analogy that is like zooming into my cortex at the moment, because there is a

02:28:32.640 perfect analogy to driving around a corner in a race car, but I will spare everybody that. You can

02:28:37.940 put it in the show notes. There we go. Well, Annie, thanks so much. This was really great. I really

02:28:41.460 appreciate you responding to my random request to sit down and talk about this. And I can't wait to do it

02:28:46.960 again when the new book comes out. Well, thank you. I'm excited to come back when the workbook is done,

02:28:50.660 but this was like an amazing conversation, very different than most of the podcasts I've done.

02:28:54.680 So I really appreciate it. Oh, that's the way we do things here. We go deep. Yeah.

02:28:59.800 Thank you for listening to this week's episode of the drive. It's extremely important to me to

02:29:04.280 provide all of this content without relying on paid ads to do this. Our work is made entirely possible

02:29:09.820 by our members. And in return, we offer exclusive member only content and benefits above and beyond

02:29:16.200 what is available for free. So if you want to take your knowledge of this space to the next level,

02:29:20.660 it's our goal to ensure members get back much more than the price of the subscription premium

02:29:25.640 membership includes several benefits. First, comprehensive podcast show notes that detail

02:29:31.500 every topic, paper, person, and thing that we discuss in each episode. And the word on the street

02:29:37.120 is nobody's show notes rival ours. Second, monthly ask me anything or AMA episodes. These episodes are

02:29:45.220 comprised of detailed responses to subscriber questions, typically focused on a single topic

02:29:50.080 and are designed to offer a great deal of clarity and detail on topics of special interest to our

02:29:55.440 members. You'll also get access to the show notes for these episodes. Of course, third delivery of our

02:30:01.340 premium newsletter, which is put together by our dedicated team of research analysts. This newsletter

02:30:06.720 covers a wide range of topics related to longevity and provides much more detail than our free weekly

02:30:12.960 newsletter. Fourth, access to our private podcast feed that provides you with access to every episode,

02:30:19.860 including AMA's sans the spiel you're listening to now and in your regular podcast feed. Fifth,

02:30:26.780 the qualies, an additional member only podcast we put together that serves as a highlight reel featuring

02:30:32.960 the best excerpts from previous episodes of the drive. This is a great way to catch up on previous episodes

02:30:38.700 without having to go back and listen to each one of them. And finally, other benefits that are added

02:30:43.500 along the way. If you want to learn more and access these member only benefits, you can head over to

02:30:49.040 peteratiamd.com forward slash subscribe. You can also find me on YouTube, Instagram, and Twitter,

02:30:56.000 all with the handle peteratiamd. You can also leave us review on Apple podcasts or whatever podcast player

02:31:02.980 you use. This podcast is for general informational purposes only and does not constitute the practice

02:31:08.680 of medicine, nursing, or other professional healthcare services, including the giving of

02:31:13.180 medical advice. No doctor patient relationship is formed. The use of this information and the

02:31:19.040 materials linked to this podcast is at the user's own risk. The content on this podcast is not intended

02:31:25.080 to be a substitute for professional medical advice, diagnosis, or treatment. Users should not disregard

02:31:30.700 or delay in obtaining medical advice from any medical condition they have, and they should seek

02:31:35.260 the assistance of their healthcare professionals for any such conditions. Finally, I take all conflicts

02:31:41.280 of interest very seriously. For all of my disclosures and the companies I invest in or advise, please

02:31:47.200 visit peteratiamd.com forward slash about where I keep an up-to-date and active list of all disclosures.

02:32:00.700 Thank you.

The Peter Attia Drive - September 02, 2024

Improve your decision-making, frameworks for learning, backcasting, and more ｜ Annie Duke (#60 rebroadcast)

Episode Stats

Summary

Transcript