The Peter Attia Drive - #60 - Annie Duke, decision strategist： Poker as a model system for life—how to improve decision making, use frameworks for learning, and apply 'backcasting' to boost your odds for future success

00:00:00.000 Hey everyone, welcome to the Peter Atiyah Drive. I'm your host, Peter Atiyah.

00:00:10.140 The drive is a result of my hunger for optimizing performance, health, longevity, critical thinking,

00:00:15.600 along with a few other obsessions along the way. I've spent the last several years working with

00:00:19.840 some of the most successful top performing individuals in the world. And this podcast

00:00:23.600 is my attempt to synthesize what I've learned along the way to help you live a higher quality,

00:00:28.360 more fulfilling life. If you enjoy this podcast, you can find more information on today's episode

00:00:33.000 and other topics at peteratiyahmd.com.

00:00:41.440 Hey everybody, welcome to this week's episode of the drive. I'd like to take a couple of minutes

00:00:46.000 to talk about why we don't run ads on this podcast and why instead we've chosen to rely entirely on

00:00:51.660 listener support. If you're listening to this, you probably already know, but the two things I care

00:00:56.480 most about professionally are how to live longer and how to live better. I have a complete fascination

00:01:02.780 and obsession with this topic. I practice it professionally and I've seen firsthand how

00:01:07.480 access to information is basically all people need to make better decisions and improve the

00:01:12.120 quality of their lives. Curating and sharing this knowledge is not easy. And even before starting the

00:01:17.600 podcast, that became clear to me. The sheer volume of material published in this space is overwhelming.

00:01:22.820 I'm fortunate to have a great team that helps me continue learning and sharing this information

00:01:27.880 with you. To take one example, our show notes are in a league of their own. In fact, we now have a

00:01:33.840 full-time person that is dedicated to producing those and the feedback has mirrored this. So all of this

00:01:39.720 raises a natural question. How will we continue to fund the work necessary to support this? As you

00:01:45.980 probably know, the tried and true way to do this is to sell ads. But after a lot of contemplation,

00:01:51.100 that model just doesn't feel right to me for a few reasons. Now the first and most important of

00:01:56.300 these is trust. I'm not sure how you could trust me if I'm telling you about something when you know

00:02:02.100 I'm being paid by the company that makes it to tell you about it. Another reason selling ads doesn't

00:02:07.580 feel right to me is because I just know myself. I have a really hard time advocating for something

00:02:12.840 that I'm not absolutely nuts for. So if I don't feel that way about something, I don't know how I can

00:02:18.500 talk about it enthusiastically. So instead of selling ads, I've chosen to do what a handful

00:02:23.280 of others have proved can work over time. And that is to create a subscriber support model for my

00:02:29.320 audience. This keeps my relationship with you both simple and honest. If you value what I'm doing,

00:02:36.260 you can become a member and support us at whatever level works for you. In exchange, you'll get the

00:02:41.840 benefits above and beyond what's available for free. It's that simple. It's my goal to ensure that no

00:02:47.860 matter what level you choose to support us at, you will get back more than you give.

00:02:52.960 So for example, members will receive full access to the exclusive show notes, including other things

00:03:00.380 that we plan to build upon, such as the downloadable transcripts for each episode. These are useful

00:03:06.760 beyond just the podcast, especially given the technical nature of many of our shows. Members also get

00:03:12.900 exclusive access to listen to and participate in the regular ask me anything episodes. That means

00:03:19.840 asking questions directly into the AMA portal and also getting to hear these podcasts when they come

00:03:25.800 out. Lastly, and this is something I'm really excited about. I want my supporters to get the best

00:03:30.780 deals possible on the products that I love. And as I said, we're not taking ad dollars from anyone,

00:03:35.620 but instead what I'd like to do is work with companies who make the products that I already love and would

00:03:40.600 already talk about for free and have them pass savings on to you. Again, the podcast will remain

00:03:47.740 free to all, but my hope is that many of you will find enough value in one, the podcast itself, and two,

00:03:56.260 the additional content exclusive for members to support us at a level that makes sense for you.

00:04:01.680 I want to thank you for taking a moment to listen to this. If you learn from and find value in the

00:04:06.180 content I produce, please consider supporting us directly by signing up for a monthly subscription.

00:04:11.700 My guest this week is Annie Duke. Annie is the author of the bestselling book,

00:04:16.280 Thinking in Bets, Making Smarter Decisions When You Don't Have All the Facts. And some of you may

00:04:20.580 recall a few months ago, I talked about this book a little bit in my Sunday morning newsletter because

00:04:26.620 it was just one of those books where even halfway through it, I realized this was a book everybody

00:04:30.740 needed to read and I couldn't get enough of it. And I immediately reached out to Annie and said,

00:04:34.400 I'd love to have you on the podcast and what follows is that discussion. But for those of

00:04:38.980 you who may have not got that or missed that, Annie was one of the top poker players in the world.

00:04:43.600 She played professionally for about 20 years. In 2004, she won her first world series of poker

00:04:48.700 bracelet. That same year, she won $2 million in a winner take all invitation only world series

00:04:53.860 poker tournament of champions. In 2010, she won the prestigious NBC national heads up poker

00:04:59.020 championship. Prior to all of that, by the way, and we talk a little bit about this,

00:05:03.280 she was actually awarded a National Science Foundation fellowship to study cognitive psychology

00:05:07.740 at the University of Pennsylvania. So poker actually is something that kind of came later

00:05:11.180 to her in life. She now spends her time writing, coaching, and speaking on a wide range of topics

00:05:16.360 from decision fitness, emotional control, productive decision groups, embracing uncertainty.

00:05:21.040 She's also done a lot of work in charity. And unfortunately, we don't get into any of that. I had planned

00:05:26.000 to, but as is often the case, we get so far into something in this case, obviously poker and

00:05:31.840 uncertainty and decision-making that after three hours or so, we hadn't even got to some of the

00:05:35.900 other things that she's done. We talk a lot about how she got into this. And of course, we do spend

00:05:41.080 some time defining poker because if you're not a poker player, and I'm not a poker player, I mean,

00:05:44.680 I sort of know the rules, but I think it is important to understand the nuances of the game,

00:05:50.440 at least as far as the rules go and the different types of poker. So even if you've never played a hand

00:05:56.520 of poker in your life and don't know the difference between Texas Hold'em and Blackjack, for that

00:06:00.960 matter, you'll still get a ton out of this because we really explain that stuff. And we contrast poker

00:06:06.280 with other things like chess, where clearly you have to have a lot of skill to play them,

00:06:12.520 but luck is not an element. We talk about why poker is in some ways the model system for decision

00:06:18.980 making in the real world. We talk about backcasting, which is something that I believe I alluded to in

00:06:26.620 my newsletter that is what really got me excited about the way she described this. And we talk about

00:06:32.160 doing pre-mortem exams versus just post-mortem exams. We go through the decision matrix, meaning

00:06:38.900 how do you look and evaluate decisions after the fact when you have to be critical of both the

00:06:44.700 decision making process and the outcome. And I have to tell you, this is just yet another example

00:06:49.820 of a podcast where I came away learning so much. And in preparing for this, I obviously have read

00:06:56.440 the books and I've thought a lot about decision making and I know quite a bit about probability

00:06:59.940 theory. It didn't matter. Annie still took my understanding of this to another level. And in

00:07:05.780 the process, I realized she is working on another book, which is a workbook to help with decision

00:07:09.860 making. That's going to be out next year. So one, A, I can't wait to get that. And two,

00:07:14.080 we're going to have to have Annie back when that book comes out and do a real deep dive 0.82

00:07:18.080 on how to implement that workbook. So I could go on for a lot on this, but you don't want to

00:07:22.800 hear me talk. You want to hear Annie talk. So without further delay, here is my guest, Annie Duke.

00:07:30.620 Annie, thanks so much for coming all the way up to New York. I'd like to say just to see me,

00:07:34.760 but now you've let the cat out of the bag that I'm the second excuse to be here today.

00:07:38.800 Yes. When I get requests, I try to pack them into the same day for efficiency purposes.

00:07:43.400 Are you from Philly originally?

00:07:45.180 I'm not from Philly, but my father is from Philly originally. So yeah, my dad grew up in

00:07:50.320 Philadelphia and graduated from West Philly High actually. So he's born and bred and I live in

00:07:56.880 Philadelphia now. It's actually my third time through. So there's clearly some kind of magnetic

00:08:02.240 center to Philadelphia for me.

00:08:04.140 Are you an Eagles fan?

00:08:05.200 I am.

00:08:05.660 I'm sorry. I can't. I'm so angry about them beating the Patriots two years ago.

00:08:10.180 I'm just so upset about it.

00:08:11.300 Oh, please. Oh, I'm so sorry for you that they didn't get their sixth Super Bowl that day and

00:08:17.120 they had to wait two more years.

00:08:19.140 I don't know. It just bothered me.

00:08:20.460 All right. So wait, are you a Patriots fan?

00:08:23.080 I am.

00:08:23.400 Okay. So interestingly enough, I am a Red Sox fan because I grew up in New England and

00:08:28.280 it's kind of just an accident of sort of the timing of my own life that I'm an Eagles fan,

00:08:34.300 which has to do with, I was never much interested in football when I was growing up, which isn't

00:08:39.300 that crazy for when I was growing up in terms of the Patriots because they were awful.

00:08:43.120 Horrible.

00:08:43.240 Oh, just horrible. It was horrible. So I liked the Celtics because Larry Bird and Kevin McHale

00:08:49.680 and obviously the Red Sox. It was like Fred Lynn and Carl Yastrzemski and Carlton Fisk and like that

00:08:55.600 amazing team. So, and we used to go like once a year. So I would like to remind you as you're so

00:09:02.040 upset that they didn't get number six. On that day.

00:09:05.620 Of what it was like to be a Red Sox fan.

00:09:08.240 I know. I know. Well, you brought up Fisk, right? So it's like, yeah.

00:09:11.060 Was it the 79 team or the 76 team?

00:09:14.240 I think it was 76, but someone's going to correct me, but yeah.

00:09:16.920 Where it's like almost.

00:09:18.360 No, I know. That's, it's hard to watch. You know what it is? I'm a Belichick fan. So people sort of

00:09:22.740 say, oh, you're a Patriots fan. Like wow, wow, wow. But it's not that I disproportionately like anyone

00:09:27.480 on that team. Although I think Tom Brady is, I think unequivocally the greatest quarterback of all

00:09:32.240 time at this point. But to me, it's just, it's Belichick. Like I'm obsessed with the way this guy

00:09:38.900 operates. Like you always get asked the question, like if you could meet anybody famous, who would

00:09:42.640 it be? I mean, at the top of my list would have to be him. Like, and again, it wouldn't be in the

00:09:47.900 setting of him giving glib answers in a news conference. Cause that wouldn't be that much

00:09:51.640 fun. Although it's funny as hell to watch. If I could spend a day on his boat with him,

00:09:56.460 really understanding the discipline that he brings to what he does, it would just,

00:10:00.300 it would just amaze me. There's a beautiful article that was written probably five years ago.

00:10:05.000 It was just told in a really funny way. Cause it was the gist of the article was everybody's got a

00:10:09.340 Belichick story and it's players, coaches, all telling these stories of him that really highlight

00:10:16.700 this single minded focus of winning. That is kind of amazing to me, even though from a life ethos

00:10:23.140 standpoint, I don't really care that much about sports anymore. I've sort of, there was a day when

00:10:27.180 it really upset me if my team lost. Now it's sort of, oh, that sucks. 0.96

00:10:30.620 Well, it's good that you've managed to come to that point. 0.98

00:10:34.540 Oh my God. Like I could tell you stories about when I was in high school, if my team lost,

00:10:38.320 it was like, if you want to get at least one degree, just one degree from Bill Belichick,

00:10:42.720 you should have Michael Mbardi on who wrote Gridiron Genius. It's a great book about leadership.

00:10:48.680 He worked with Bill Walsh, he worked with Belichick. He really kind of talks about what the

00:10:54.660 leadership qualities of those two particular coaches. He's really great. You should look him up.

00:11:00.200 Do you know him?

00:11:00.620 I do, yes.

00:11:01.140 Well, then I'm going to ask you to make that introduction.

00:11:02.980 I will make that introduction.

00:11:04.200 I met Bill Walsh about a month before he died and it was the most sort of simple, he was at

00:11:10.700 Stanford and I guess he was getting some medical care and I was there. I lived in the Bay Area at

00:11:15.620 the time, but I still swam at the Stanford pool. And one day I was just finishing a swim and I was

00:11:20.500 walking back to the parking lot and I can't believe it, but Bill Walsh is like walking alone,

00:11:25.060 like on this part of the campus. I normally would never interrupt somebody to be the fan boy

00:11:29.780 because I just feel like it's sort of, I don't know, that's not my style to do it,

00:11:32.940 but I couldn't help it. And I was, oh my God, I just want to shake your hand. And of course,

00:11:37.860 like a total idiot, I just started telling him all of my favorite moments of his coaching career 0.99

00:11:42.900 as though he doesn't remember them. Like, do you remember the drive at the end of the second 0.99

00:11:48.780 Bengals Super Bowl when there were only two minutes and four seconds left? And remember when Taylor

00:11:53.620 and he was just the kindest, he totally entertained all of my nonsense. Like, yeah,

00:11:59.040 those were some great times. Those are some great memories. And sadly he died like a month later.

00:12:03.580 I mean, he was sick in that moment. He was totally able to, anyway. So, but that said,

00:12:07.400 I'm not happy about the Eagles and we'll just let that slide.

00:12:11.420 Literally, it was like two years later. Here's the thing. Belichick is in no danger of not getting

00:12:16.840 into the Hall of Fame. Now there's Tom Brady. This is what I'm trying to remind you of the Red 1.00

00:12:23.000 Sox, letting the Eagles have their one moment. Yeah. Well, now the Red Sox can't stop doing it.

00:12:27.960 It's also sort of. Yeah. Cause I'd like to remind you that the Eagles did get their first try in like

00:12:33.120 ever at the Super Bowl against the Patriots in the first place in loss. So like, come on,

00:12:38.520 it's like crumbs, crumbs for the peasants. 0.93

00:12:42.720 So first of all, I read your book. I don't know how long ago it's been, but I mean, it was

00:12:46.280 probably 10 minutes before I reached out to you on Twitter. I loved it. And I think we all sort of

00:12:50.940 read books and apply our own filter to them. And of course, one of the filters is on this business

00:12:56.160 of decision-making and in medicine, we make decisions all the time and we make decisions

00:13:01.560 within complete information. And I remember explaining to somebody once something that I

00:13:06.180 think I either heard you talk about on a podcast after the fact and describe it more eloquently.

00:13:10.340 And I think you also alluded to it in the book, which is people always think of chess as this perfect

00:13:15.460 game for decision-making. So actually my wife, I was having this discussion with, I said, no,

00:13:19.820 actually chess is not a great example because everybody has perfect knowledge at every moment

00:13:23.820 in the game. So it's not to discount the importance of strategy in chess. People who are brilliant at

00:13:29.080 chess are brilliant, but it's not the perfect analog for decision-making in life. For that,

00:13:34.600 poker is much better because you have incomplete information and that tends to mimic what we see in

00:13:40.340 life. And in medicine, I think you almost never have complete information. And that's sort of one

00:13:45.600 of the things that is both frustrating because how can you make a serious decision when you don't know

00:13:51.180 all of the facts, but at the same time, I think makes it intriguing and interesting and gives you

00:13:56.520 this sense of there's a craft that can be mastered here that is less sort of amenable to robotic

00:14:02.340 decision-making.

00:14:03.120 Tess is missing this particular element, which decision-making in general has,

00:14:08.100 which has to do with incomplete information. So in chess, I can see the player's whole position.

00:14:13.720 And what that means is that I should be able to calculate out if I'm considering a move,

00:14:18.060 I know what all the possible moves are that the player can make. And then I can think of what

00:14:21.940 all the possible moves are that I can make in response and so on and so forth, so that I should

00:14:25.800 be able to calculate out with absolute perfect knowledge here, what the right move is at any given

00:14:30.640 time. It's missing another element though, that I think is really important to think about,

00:14:35.060 which is a very strong influence of luck. So in chess, the pieces stay where they are. Nobody

00:14:42.300 takes dice, you roll an 11, and two pawns come off the board, or if it's snake guys, checkmate.

00:14:49.020 Or maybe something good happens, you roll a six and you get an extra queen. That doesn't happen in

00:14:55.140 chess so that there's this, we know that the pieces are only ever going to move through an act of skill.

00:14:59.940 And this idea of the strong influence of luck is really, really important. Meaning, if I were to

00:15:06.940 do the same thing in the same situation, and then post decision, are there other things that could

00:15:13.200 reasonably happen that I don't have control over? I don't have control over which thing is actually

00:15:17.860 going to occur. It's actually a really important element to the way we make decisions. If you think

00:15:22.340 about it in poker, if I put my money in with aces, and you put your money in with fives, you know,

00:15:28.820 80 ish percent to win there, slightly better. That means 80% of the time I'm going to win. But that

00:15:34.420 doesn't mean that you can't win 20% of the time you can win. And I have no control either way. I don't

00:15:39.900 know if on that particular iteration, I'm going to see the 80% or the 20%. That really crosses all

00:15:46.140 decision making. So you can tell me this would be my intuition as not a medical practitioner. But

00:15:52.140 not only are you dealing with incomplete information in medicine, but even if you did have

00:15:57.380 complete information, if you apply a treatment in a situation, and it happens to work, there's all

00:16:03.840 sorts of stuff going on, I assume that's relatively stochastic, that we don't really understand,

00:16:08.660 where if we were to do that same thing again, it may not work in the next situation. So we always

00:16:13.720 want to think about that, because that has a very big effect on our decision making.

00:16:17.360 Yeah, you're absolutely correct. And just to hear you explain it that way, that might even be the

00:16:21.660 more important factor, actually, than the absence of complete information. Now, the definition of a

00:16:26.880 chaotic system versus just a stochastic system is even more profound, where even a slight change in

00:16:31.980 the initial condition can produce a profound change in the outcome. We don't even have to posit that we

00:16:37.460 can just leave it at the same initial conditions can produce different outcomes, right? We can kind of

00:16:42.920 unpeel it in this particular way. Let's say that we're dealing in a world where we had perfect

00:16:47.920 information. I think it's important to think about this to understand why chess is really so bad in

00:16:53.360 terms of thinking about these kinds of decisions. It's an interesting decision making problem. I'm not

00:16:58.080 trying to slam chess, I think it's a great game. And it teaches critical thinking skills. And it teaches

00:17:02.620 you to think many moves ahead and to think about what other people know. And there's all sorts of great

00:17:07.520 benefits that it has, it just happens to be missing a couple things that I think are really

00:17:11.040 important. But let's say that we had perfect information. For example, if I have a coin,

00:17:16.480 and I've weighed it, so that I know that the weight is correct, I've examined it, and I can see that it's

00:17:24.260 two sided, then I now have perfect information. I know exactly what I need to know about the coin

00:17:31.260 to understand how often it's going to flip heads and how often it's going to land on tails. But what I

00:17:37.180 can't know is what it will land on the next try. So that's that issue of luck. So if you flip and

00:17:43.720 it lands heads, it's reasonable for me to say it could have landed tails. And if we did that again,

00:17:50.080 that tails is totally a possibility, regardless of what I happen to have seen on this particular

00:17:54.360 flip. So and then what we can do is say, okay, so even in conditions of perfect information, we don't

00:18:01.180 really know which outcome we're going to get, we might know the frequency, how often that might occur,

00:18:06.180 or how often we could expect it to occur, but we don't know for sure what will happen this time.

00:18:10.700 And then you can take it further and say, but what if you haven't examined the coin?

00:18:14.700 And based on some set of outcomes, you're now trying to derive what the coin looks like.

00:18:20.220 Now that becomes even harder. And that's where you get into something that looks very little like

00:18:25.920 chess. Why is it do you think that as humans, we, at least most people, most of us don't seem

00:18:33.460 innately wired to understand probability theory because I studied math as an undergrad and I

00:18:39.360 spent so much time in it. I think I take for granted now how easy it is for me to think

00:18:44.840 probabilistically. I sort of view the world as a stochastic map, but I'm pretty sure I wasn't born

00:18:50.240 that way. And I'm pretty sure that nothing, maybe this isn't correct, but I doubt that we were

00:18:55.180 evolutionarily wired to do that. Or I feel like we would see it more often in the way people behave.

00:19:00.600 And I feel like the absence of that in the general person's point of view, who hasn't been forced to

00:19:05.920 be trained in that way, speaks to that. This strikes me as one of the more difficult things in

00:19:10.400 explaining anything. This touches everything from politics to business to science, because it's too

00:19:18.400 easy to say, well, that didn't happen. Therefore that couldn't have happened as opposed to describing

00:19:25.200 it the way you did, which is no, actually it could have happened. And if the same initial

00:19:28.300 circumstance happens again, by the way, this thing could happen.

00:19:30.780 I would imagine if we were to take the distribution of how naturally probabilistic thinking comes to

00:19:38.180 somebody, I'm guessing that you were probably born closer to the right tail. That being said,

00:19:43.820 I guarantee you that if I followed you around for a day, I could find all sorts of spots where you're

00:19:48.740 not thinking probabilistically. And I could in particular find all sorts of spots where your initial gut

00:19:55.380 reaction is not probabilistic and you have to stop yourself.

00:19:58.820 I'm positive. I'll give you an example of a game I played. That's not an exact example of this,

00:20:04.260 but it humbled the hell out of me. And I always go back and play this with other people. So the idea

00:20:10.100 of the game was to teach people what a 95% confidence interval looks like. And this is back when I was at

00:20:15.360 a consulting firm called McKinsey and company. And I was a part of the risk practice. Our whole job was

00:20:19.460 walking around and helping companies manage risk. So we did this at an offsite one day to help us kind of

00:20:26.040 figure this out. So the moderator goes up with 20 questions. Have you played this game?

00:20:30.600 So I have, and this is one of Phil Tetlock's favorite who wrote expert political judgment and super

00:20:36.380 forecasting, which I highly recommend anybody interested in probabilistic thinking.

00:20:40.200 I ended up reading super forecasting years after this experiment.

00:20:43.420 So you know, he's got this in there. Yeah.

00:20:44.960 So I'll tell it just for the listener. So you take, and I still have it, by the way,

00:20:47.680 I still have the 20 questions. I was terrible at this.

00:20:49.960 I'll tell you what my score was after. I'm so pissed about it too. Like how bad I was.

00:20:53.580 Mine was like zero. So go ahead. And I've really tried. It's just like the questions that I got

00:20:59.060 were really out of my knowledge base. And it was like, I was trying to have really wide,

00:21:02.660 but it was a disaster. Okay. So what the moderator does is they give you 20

00:21:06.200 questions. Each one has a quantitative answer that is known. That is known. Yeah. And the question for

00:21:12.980 you, the participant is for each question, give me a bracket that contains the answer. So for example,

00:21:20.540 what is the average height of a male in the United States? An appropriate answer would be

00:21:27.440 somewhere between four foot six and six foot one, because that would be a reasonable guess

00:21:34.380 at a bracket that contains the average height of a male.

00:21:38.040 Right. So the way that I've heard it is that you want the bracket to contain it 95% of the time.

00:21:42.700 Which means after you've done all 20, you should have exactly one incorrect and 19 correct. And the

00:21:51.040 purpose of saying 95% confidence is if they said to you, you have to have a hundred percent

00:21:56.320 confidence. You can sandbag it. One inch tall. Yeah. There's somewhere from one inch tall to a

00:22:00.780 million miles tall. And the questions in fairness to people like you and I who have flunked the test,

00:22:05.340 the questions are hard. So did you get, how many votes did Abraham Lincoln receive? I didn't get that

00:22:10.360 one. I got like, what is the distance between Jupiter and its nearest moon? It's like, I don't

00:22:15.460 have any goddamn clue, but how many people were voting during Lincoln's? I don't know. And I had 0.97

00:22:22.520 this really wide range, you know, it's great, but it's such a great exercise because you think, okay,

00:22:28.320 I've got this under control. I'm going to just try my best to get everyone correct with a reasonable

00:22:34.820 guess and I'll probably get one wrong. And I think in the end I got 12 right and eight wrong, which of

00:22:40.740 course is like a 60% confidence interval on things that are perfectly knowable. And so, yes, I completely

00:22:47.820 agree with you. And of course your initial example is a better one, which is how many times do I make

00:22:53.100 either a decision or have a gut reaction that I have to actively talk myself out of?

00:22:59.600 The reason why I bring that up is that I think that there's a couple of things that we want to

00:23:03.220 sort of keep in mind. And then if you want to talk about sort of like from a, like an evolutionary

00:23:07.500 story, why it is that we're not particularly wired to think probabilistically, we can get into that.

00:23:13.180 But what I want to get across is this, that first of all, we're all sort of like sitting,

00:23:17.860 we have proclivities, you know, where we're sitting on a distribution, somewhere on the distribution in

00:23:22.860 terms of sort of how we're born, how we're wired to think probabilistically. And that doesn't mean

00:23:29.660 that we should say either we're there or we're not. I mean, the idea is that wherever we sit on

00:23:35.200 the distribution or whatever our personal distribution looks like, we just want to shift

00:23:39.620 it to the right. So wherever you are, you want to accomplish two things. One is it would be really

00:23:46.240 good if you thought probabilistically a little bit more. Why? Because the world is probabilistic.

00:23:53.980 And so the more accurately you can think about what the outcomes of your decisions might be,

00:23:59.720 because they're not deterministic. When you make a decision, there's many, many things that could

00:24:04.680 occur with some likelihood of each of those things occurring. And understanding that particular fact

00:24:09.040 is incredibly important to good decision making. So the more that you can shift yourself a little bit

00:24:13.260 more on the side of probabilistic thinking, the better off you are, no matter where you started.

00:24:17.260 And then the second thing we want to accomplish, which is what I was trying to get at,

00:24:20.040 at the catching yourself part, is to understand that we are wired this way. So it's really hard

00:24:26.960 at all times to think that you don't know things for certain, to sort of live in this counterfactual

00:24:34.460 world. And it's frankly, kind of impractical, it would be hard to like drive, for example, if we were

00:24:39.460 living counterfactually all the time. So the idea is that when we do make an error to catch it faster,

00:24:45.780 because for most things, when we make an error, and we we have too much confidence, or we're thinking

00:24:52.140 about something as deterministic, when we should be thinking about it as probabilistic, for most

00:24:57.140 things, we don't catch it at all. So if we can just catch it a little bit more often, and if we can

00:25:03.060 catch it a little bit more quickly, we're a lot better off. And these small shifts, if you can get a

00:25:09.560 couple percentage points better, you're way better off in your decision making.

00:25:14.700 So let's go back to kind of where you started thinking about this, which was through the lens

00:25:18.800 of playing competitive poker. How did you end up playing poker?

00:25:22.580 Chaos theory. So let me just say that when I started playing poker, it wasn't all over television,

00:25:27.540 because I think that otherwise, that would be a really simple answer. Like I was in college,

00:25:31.160 and I wanted to make money. And I saw poker on TV, and I started playing online,

00:25:35.860 I think would be the origin story of the majority of poker players today. Not my origin story,

00:25:41.760 because I'm way too old for that.

00:25:43.300 Where did you go to college?

00:25:44.140 Columbia.

00:25:44.620 Okay. Yeah.

00:25:45.320 So you're here in the city.

00:25:46.340 So I'm here in the city, and there's no internet. I mean, I think there's some sort of, I don't know,

00:25:51.720 Al Gore was inventing it, or something. I don't know, but it didn't exist. So the first thing that

00:25:56.740 happened on my way to becoming a professional poker player is that my brother, when he was in high

00:26:02.940 school, got very, very interested in chess, started really studying it, became incredibly good at the

00:26:08.560 game. He got accepted to Columbia, decided to take a year deferment to live in New York and study with

00:26:16.680 a grandmaster. During that year, he started playing poker. All sorts of things happened. He lost his

00:26:23.140 college nest egg. He eventually did very well. But in this particular year, he was learning,

00:26:29.760 let's say. And he was learning maybe on the money that had been saved for college. It was a little

00:26:34.640 bit of money. It wasn't a lot. Anyway, he had started playing. And then I followed to New York

00:26:39.640 to go to Columbia a year later. I actually went. I didn't defer. And by the end of that year, my

00:26:45.040 brother was very good. So now we're both living in New York at the same time. He actually doesn't

00:26:49.940 end up going much later. He ended up going, I think, for like six months, but he ends up never

00:26:54.280 finishing college. And he's playing poker professionally. And I'm going and watching

00:26:59.200 him sometimes. So I'm just sort of sitting behind him. And what does that mean at this

00:27:02.520 era of time? These are like smoky rooms in New York City? Yeah. There was a place called the Mayfair,

00:27:08.120 which had a game. And they had poker and they had backgammon. Before that, my brother was playing

00:27:15.920 at a place called the Bar Point. For people who don't know, that's a backgammon term. And it was like

00:27:21.940 a backgammon club. People also played gin there. It was on 14th and 6th. And in the back room,

00:27:28.940 there was a poker game. This was before I came around. He was sort of already out of this place

00:27:33.060 by the time that I came around. Although I did spend a summer here where I got to see him sort

00:27:37.500 of playing in that particular place. So he found out about that through like his interest in chess,

00:27:43.420 ends up there in the back room. There's a game that starts on Friday night and ends on Monday morning.

00:27:48.000 And by that, I mean, that's how long the people played. They didn't get up and leave and come

00:27:53.880 back. We can have a whole other discussion about sleep deprivation and decision making,

00:27:57.880 but we'll save that for another time. Yeah, this was in the early 80s. So I believe there was

00:28:01.820 cocaine involved. Yeah, I'm sure. But at any rate, so he starts playing in that game. And that's

00:28:07.080 actually where he... So he cuts his teeth. That's where he's cutting his teeth. That's where he's sort of

00:28:10.640 learning how to play. That's where he's losing this college money. And eventually, so he's playing

00:28:16.620 in that game. He's not doing very well. And he starts figuring out, oh, actually, it's sort of in

00:28:22.080 the same way he approached chess. He sort of found whatever books he could find. And he found a few

00:28:26.960 other people who were kind of really interested in learning the game. And he sort of starts working

00:28:30.940 the game. And eventually, he basically sort of breaks that game. Like he becomes so good

00:28:37.840 that I can't remember whether they kicked him out or not. But in the meantime, when he had lost his

00:28:42.900 college money and what he starts doing, because now he needs money to be able to play, is he became

00:28:48.220 like the errand person for the game. And so they'd be like, hey, Howard, will you go get me a sandwich

00:28:54.800 like at the deli downstairs or go get me some pizza or go get me cigarettes or go whatever. And he'd go

00:29:00.380 and do an errand and then he'd get a tip. And then he'd save his tips up and then get into the game.

00:29:04.800 Eventually, he didn't need to be the errand person anymore, because he'd turned the tip money into

00:29:09.140 enough money for him to eventually send himself to college for that brief period.

00:29:14.120 But so he moved from there to the Mayfair, which was

00:29:16.480 And the Mayfair is where?

00:29:17.460 Is it in Gramercy Park?

00:29:19.320 Okay.

00:29:19.620 Somewhere near there? It's either Gramercy or Chelsea. I think it's Gramercy Park. I think

00:29:24.120 it's near Gramercy Park. It's a nice place. Then he was playing there for a while. And he falls in at

00:29:28.500 that point with a group that really becomes very famous. Dan Harrington, main event champion.

00:29:35.580 Eric Seidel, who I think has nine world championship bracelets from the World Series of

00:29:40.640 Poker. It might be more than that now. But he's won, I think, like $40 million playing poker. I mean,

00:29:44.700 he's amazing. Steve Zolotow, also bracelet holder, was in that group. Jason Lester.

00:29:50.820 And these guys are all coming out of this one group. So this is sort of like Daniel Coyle

00:29:55.140 writes about these centers of excellence, or I forget what his term is for them. But these

00:29:58.940 little places that just produce so much greatness, like all these soccer players that come out of

00:30:04.020 this place in Brazil, and all these tennis players that come out of this place in Russia,

00:30:08.240 this became the place where everybody's sharpening everybody in this one place.

00:30:13.620 If you're a tourist and you hear about this game, you don't get to go and play, obviously,

00:30:17.080 right? Like they'll take your money if you want, but...

00:30:18.860 Yeah, anybody can play. And anybody does play. The thing about poker that's part of this stuff

00:30:24.480 about like the hidden information in luck is that that kind of uncertainty leaves open the

00:30:30.200 array of explanations for why you might not be doing well. So if I play chess against you a few

00:30:36.380 times, it becomes very clear to me why I'm losing. I'm losing because you're better than I am. I have

00:30:41.400 nothing else to protect my ego. Like I can't put a hat on it and say, no, it's because of something

00:30:46.400 else. See, that's such a great point that's worth pausing on it as well for a moment. There are very

00:30:51.800 few sports or activities that we play that are as pure as something like chess, where as you said,

00:30:59.620 there is zero excuse. Whereas I can't think of another activity. I'm sure there are several where

00:31:07.880 if you really want to, you can't, oh, the ref screwed me or, oh, the wind. I mean, did you see

00:31:12.960 that? Or like, I'm into archery, right? It's like, oh, did you see that animal move out of the way at the

00:31:16.880 last second? He heard the arrow coming or... Whereas in reality, no, you took a lousy shot.

00:31:21.240 Right. I mean, they're all on a spectrum. And one thing that we forget is that there's an

00:31:25.600 equanimity to luck. So if you're playing tennis and there's wind, your opponent is also playing

00:31:30.880 with that wind. If you're in a game, although we like to think so, the refs generally are making

00:31:36.340 bad calls, just sort of period, some good, some bad. And they're not like screwing a particular team,

00:31:43.440 systematically going against a particular team, although it sort of feels like that.

00:31:48.100 But what we notice is the wind blew our ball. An interesting thing that happens in sports that

00:31:53.220 are one-on-one, like I play tennis, is that you can blame it on the style matchup. That it's not

00:31:57.720 that you're not particularly good. That's a great new excuse. I hadn't even really thought of it.

00:32:00.880 No, you should use that. Like, oh, that was a bad style for me. I mean, I'm actually better than

00:32:04.760 them, but they're lobbing. They were doing a lot of lobbing and my high balls weren't really on that 0.95

00:32:09.980 day, but I would crush anybody else. So yeah, exactly. 0.88

00:32:12.900 So poker basically is just a fertile ground for that thinking.

00:32:17.620 Right. Which means that it doesn't cause you to kind of self-select out. As you said,

00:32:22.520 this was like a nest of what was going to become the best players in the world. And of course,

00:32:26.680 people were still coming back. So he ends up at the Mayfair.

00:32:29.980 Well, I guess there's another element here, which is the variable reinforcement of winning.

00:32:35.040 Wow.

00:32:35.480 So then you've got the whole dopamine side of this coupled with this psychology of I can post hoc

00:32:41.980 rationalize my way all day long. I can rationalize my losses, pump up my winnings from an ego

00:32:48.600 standpoint. And the next win is coming at a point I can't predict. I mean, that is a recipe for

00:32:54.460 get me into this.

00:32:56.280 Actually, let me just bite into that for a second. So this is actually really interesting. So let me just

00:33:00.940 say I didn't play poker at all in college. Like I never played. It wasn't until graduate school

00:33:05.820 that I played three times or something. So I go to graduate school at Penn. I'm studying cognitive

00:33:10.100 science. This becomes important to me later as I'm thinking about the game of poker, this issue that

00:33:15.440 you just brought up, the difference between a variable reinforcement schedule and a fixed

00:33:20.540 reinforcement schedule. What we can think about is we can talk about ratio or interval schedule.

00:33:25.740 So ratio schedule would be by number. So if you put a rat in a Skinner box, and for those who don't

00:33:33.160 know, Skinner box, it's got the lever. And then either there's positive reinforcement, like a food

00:33:37.560 pellet comes out, or maybe negative reinforcement, like a shock. So let's make it positive and say it's

00:33:42.200 a pellet. A ratio schedule is by number. So it would be like every fifth press, the rat gets a pellet.

00:33:49.980 And an interval schedule is by time. Every five minutes, the rat gets a pellet. And if it's fixed,

00:33:57.420 what it means is it's actually every five presses or every five minutes. And depending on whether it's

00:34:03.400 ratio or interval, you get different behaviors from the rat. So what happens on the ratio schedule

00:34:10.620 is that the rat has this very even pressing that goes on. This very even, somewhat quick, but not

00:34:17.920 mad. Press, press, press, press, food. Press, press, press, press, press. And then if it's every five

00:34:24.980 minutes, the rat twiddles its thumbs until it's almost five minutes, and then it goes crazy. So you get

00:34:31.760 this burst of activity. It's almost like they don't want to miss a second. They want to make sure that

00:34:37.620 they get that food exactly. They don't want to give up one second where they could have had food.

00:34:42.520 And so you get this big burst of activity. Then you can also take a ratio or interval schedule,

00:34:47.000 and instead of it being fixed, you can make it variable. And what that means is it's now on

00:34:51.240 average is the way that you can think about that. So let's say on average, every 20 lever presses,

00:34:56.520 you'll get food. Or on average, every five minutes, you'll get food. So now, instead of this big

00:35:04.420 burst of activity on the interval schedule, on average, every five minutes, because they're not

00:35:09.260 sort of waiting. And then, oh, I think it's about five minutes, and they go. Now they're pressing

00:35:14.160 pretty steadily the whole time, because they're not sure anymore, right? Is it going to be the second

00:35:18.860 or not? And then on the variable ratio schedule, you get the same thing. You get a lot of pressing

00:35:24.500 very quickly, because they don't know, is it the next press or not? So that's all kind of interesting.

00:35:28.840 And you'll see a different activity, depending on whether it's variable or not. What's interesting is

00:35:33.320 when you try to extinguish the behavior. This is what's really key. So when you extinguish a

00:35:38.620 behavior, what you do is you just withdraw the reward. So now you've trained the rat,

00:35:44.160 it's pressing the lever, and then you just stop giving the rat food. And the question,

00:35:48.140 what does it do? When it's fixed, it stops doing the thing super fast. It's like, hey,

00:35:54.260 wait a minute, like I've been pressing 100 presses of this stupid lever, you haven't given me any food, 0.99

00:36:00.360 I give up. I have figured this out. There's no food coming my way. Same thing if you're doing 0.99

00:36:06.140 it by time. But when you make it variable, they just don't stop. And it's particularly bad when

00:36:11.980 it's a variable ratio schedule. So I think about it like this. So anybody who's ever watched people

00:36:19.200 sitting at a bank of slot machines in Las Vegas, those are on a variable ratio schedule that you'll

00:36:24.780 get a reward, it's going to be on average, a certain number of plays, you could get a reward

00:36:30.420 five plays in a row, you could go 80 plays without really getting much in return. And you kind of don't

00:36:36.880 know, because it's on average. And what happens to people sitting at those slot machines is that

00:36:42.340 they'll be going along and there'll be no reward. And you'll hear them say a thing. I'm due.

00:36:48.320 I'm due. And what that means is that it doesn't matter what you do to that machine, you could just

00:36:55.860 turn it off. What they think is no, it must be the next press, it must be the next press. And this is

00:37:02.200 exactly what happens to the rats in the Skinner box, you train them that on average, every 20 presses,

00:37:08.380 they get food. And that means sometimes they're getting three pellets in a row. And sometimes they're

00:37:13.100 going 80 presses without getting food. That's obviously out at the tail, no pun intended,

00:37:18.460 but it's just an average. And now you would draw it and they just never stop pressing the lever.

00:37:24.260 And it looks just like what humans do. And I sort of always imagine the little rats in the cages saying

00:37:29.060 I'm due. So this is exactly the way that something like poker works. If we're thinking about sort of an

00:37:35.740 edge, what that means is that over time, I'm going to get a certain percentage of your money.

00:37:43.440 But in a particular moment, you could win a lot of hands in a row. In a particular moment,

00:37:49.660 you could not win so many hands in a row. And unless you're a really good statistical aggregator,

00:37:55.120 right, unless you're sort of stepping back and really looking at what does this look like over

00:37:59.880 the long run, it's really hard to see what the underlying distribution is. It's hard to see whether

00:38:04.300 you're winning or losing to that situation. Well, there's a slight difference, right? Because if I

00:38:07.940 understand correctly, the slot machine is driven by an underlying probability distribution curve.

00:38:14.240 So the person sitting there playing it like the Skinner rat is kind of right in that they are due,

00:38:21.620 they just don't know when. Whereas the person playing poker, there is no certainty that they

00:38:26.380 will ever win again necessarily. In other words, it's both luck and some skill. So they could take

00:38:34.120 probabilistic good hands and misplay them and lose on a potential hand that could have been their due,

00:38:40.840 right? Yes. I'm thinking about that. Yes.

00:38:43.060 Yeah. So it's sort of like in some ways, the person at the poker table has an even harder

00:38:47.860 problem than the person at the slot machine. Yes.

00:38:51.920 Because the slot machine is pure luck.

00:38:53.360 So the slot machine simplifies the problem because the machine is set to a particular distribution

00:39:00.380 and the probability is set about how much you're losing per dollar. And so if I sit down,

00:39:09.540 the underlying probability is knowable. That's right. And your only task is to pull the lever or

00:39:15.960 push the button. Whereas in poker, you still actually have to do what we're going to get to,

00:39:21.480 which is the skill of this game. Right. And what you're trying to figure out is,

00:39:26.240 I mean, obviously, this isn't true of everybody. But when people sit down at a slot machine,

00:39:30.340 people aren't thinking I won because I'm so skillful. I mean, occasionally they are if they

00:39:34.060 think they have a system. I don't think that's true. Most people are saying like, I'm doing this

00:39:37.920 because I'm going to get lucky and maybe I'm due and whatnot. So I think that you have less

00:39:42.120 illusion about it. So now in poker, you lay on top of the fact that it's stochastic,

00:39:47.820 that obviously it's a skill game. And in the long run, with enough iterations,

00:39:53.640 your results will be solely determined by skill, right? Because the probability starts to approach

00:39:59.800 equal distribution for everybody, assuming there's no cheating.

00:40:02.820 Well, it would be true even if there's cheating for this reason, that what that would mean is that

00:40:07.740 whatever skill elements that you were applying, you wouldn't be able to overcome

00:40:11.120 the situation. So all I mean by this is that you have an expected value,

00:40:15.460 that expected value is determined by your play compared to your opponent's play.

00:40:21.100 And when you say skill, do you mean two skills? The first being the ability to look at all the

00:40:29.520 cards that are available and calculate the probability of cards that are not being shown

00:40:35.600 and the skill of being able to bluff? Or are you referring more to the latter than the former?

00:40:41.740 Skill would be the umbrella of all the things that are within your control in the game that have an

00:40:47.980 effect on the outcome. So first of all, let me just step back a second. Sometimes people think that a

00:40:54.280 game isn't skill, if in the short run, there's a really strong influence of luck. So people will say,

00:41:01.700 well, baseball is a game of skill, but poker isn't a game of skill. Because they're looking at,

00:41:07.240 well, but in poker, I could have the best hand, I could play it perfectly, and I can lose to you

00:41:14.540 because of the turn of a card. So therefore, isn't it just luck? And how can you tell if something is

00:41:20.840 a game of skill? Because I'm obviously making the assertion that in the long run, your results are

00:41:25.320 going to be solely determined by skill.

00:41:27.240 I think the way you said it actually is probably the best definition I've ever heard, which is

00:41:30.200 in the long run, if two different people can play the exact same game over and over and over again,

00:41:35.540 and have dramatically different results under the same stochastic input. Doesn't that imply that

00:41:42.280 there is a difference that is in the control of the player? And isn't that the definition of skill?

00:41:47.480 Yes. Because of actions that they take, their distribution is different.

00:41:51.100 Right. So for example, if you and I made up a new game, which would be a really boring game called

00:41:54.680 Toss Two Dice, and the person who gets the higher number wins. If you and I played this game for the

00:42:01.760 next week, we sat here at my table, and we played over and over and over again, in the end, we would

00:42:07.420 end up the same.

00:42:07.820 There'd be nothing that we could do that would change the outcome.

00:42:09.840 And therefore, there is no skill in that game.

00:42:11.880 There's no skill in that game. But if we can bet on it.

00:42:14.160 That's right. And baseball, of course, is such an obvious example of skill. Because while there is

00:42:18.940 luck involved, within a matter of minutes, you would tell the difference between me and a

00:42:23.740 professional baseball player doing anything on a baseball field.

00:42:26.660 There's a really great test that you can do to figure out if something is skill. And then let

00:42:31.960 me just get back to baseball for a second to show what's happening, why people are sort of

00:42:36.540 misperceiving with poker, whether it's a skill game or a luck game. So the first thing is you just

00:42:41.540 want to apply a test. Can you lose on purpose? Assuming you followed the rules of the game. Now,

00:42:46.100 it's not symmetric. I cannot apply a test of can I win on purpose, but I can apply a test of can I

00:42:51.880 lose on purpose. And I'll tell you why. So tic-tac-toe, let's agree that that's a skill

00:42:55.720 game. It's a bottom feeding skill game, but it is. Right. But if you and I play, neither of us can

00:43:01.820 win on purpose. Assuming that we both know how to play the game, because we're going to tie every

00:43:06.220 single time. But one of us can lose on purpose. Very easy to lose on purpose at that game.

00:43:10.580 So we don't want to apply the win on purpose, because that's hard, particularly as you narrow the

00:43:14.340 skill gap, then you can't do that. But I can lose on purpose. And in a game like baseball,

00:43:19.040 you can lose on purpose. In a game like basketball, in a game like chess, in any of these

00:43:24.820 games, you can lose on purpose. We can think of other acts of skill. Like if you're a salesperson,

00:43:30.260 you could make it so that on purpose, you never closed a sale. As an example, it's a game of skill.

00:43:36.060 If you were a lawyer, you could probably make sure you lost every case. I mean, hopefully you don't do

00:43:40.580 this. And in poker, you can indeed. I can lose on purpose very easily. I can't win on purpose,

00:43:45.980 but I can lose on purpose. And that's true of baseball as well. Somebody can lose on purpose,

00:43:50.300 but they cannot win on purpose. That you do not control. That at a basic level is a really good

00:43:56.900 test of whether something is a skill game or something is a luck game. Where I think the

00:44:01.020 confusion comes in is that as you narrow the skill gap, luck starts to play a much bigger role. It

00:44:08.440 starts to kind of show itself that influence of, hey, there's different ways that this could turn out.

00:44:14.700 And sometimes you're going to see one thing and sometimes you're going to see another thing and

00:44:18.880 you kind of don't have control over which thing you happen to see a lot of the time.

00:44:22.580 We start to see that influence as we narrow the skill gap. So that's a great point, right? That's why

00:44:27.820 you look at virtually every Superbowl these days. Most of them come down to the last series,

00:44:33.820 the last possession. I don't remember these stats, but it could be as high as like two thirds of Superbowls

00:44:39.100 come down to the last possession. And if you think about that, you could easily do what I do

00:44:44.660 when looking at the Eagles beating the Patriots and say it came down to that one stupid play when 1.00

00:44:50.100 Brady went back for the pass and boom, the ball gets stripped out of his hand. And it's like, 1.00

00:44:53.740 if that didn't happen, he absolutely would have connected with Gronk. They absolutely would have

00:44:58.140 won. And whether or not I'm being delusional or not, you can't deny the point that those teams were

00:45:02.980 so damn near perfect. If you took the Red Sox and you had them play a Little League team,

00:45:09.940 there'd be basically no influence of luck, right? Like what luck is going to go the way

00:45:14.160 of the Little League team? I mean, with the Red Sox, what luck is going to go against them?

00:45:19.980 Aside from all their players all at once get injured and can't play anymore, they could put in

00:45:26.340 their third stringers and the Little League team is crushed. But if you put the Red Sox,

00:45:32.660 against the Yankees, this is why we have to have seven games. And it's probably still not enough

00:45:40.400 to try to figure out who's supposed to advance to the next round. And in fact, we know it's very

00:45:45.860 rare in a series that a team wins for nothing. It's very often going to the sixth or seventh game.

00:45:52.800 Why? Because it's very narrow. And it's very often things like you just said, who wins the coin toss

00:45:58.000 in a football game? That has a huge effect on what the outcome is. Who just gets to go first?

00:46:02.400 Oh, especially in overtime.

00:46:03.520 Overtime. Yeah.

00:46:04.260 Oh my gosh. That's a huge influence of luck. Why? Because you've narrowed the skill gap so much.

00:46:08.900 But if you took that NFL football team and you put them against peewee players.

00:46:14.280 Or even a college team.

00:46:15.380 Or even a college team.

00:46:16.300 It's unbelievable that gap.

00:46:17.380 The coin toss would have no effect. No one would care who won the coin toss in overtime. The NFL team,

00:46:24.280 I mean, I don't know how they would get to overtime, but you just magically make overtime appear.

00:46:28.720 You put them in there and the coin toss will have no effect whatsoever.

00:46:32.980 So I think this is something for people to really understand. And so if we think about a game like

00:46:37.240 poker and a lot of these decision-making processes that we do in life, the skill gap is actually quite

00:46:41.880 narrow. So once I've taught you in poker things like, you know, the ranking of the hands, like a

00:46:48.520 flush beats a straight at you. You know, you can bluff, you know, sort of about betting. You can read your

00:46:54.180 hand. All these things. That's so much of the building blocks of the game in terms of the skill.

00:46:59.060 And how long does it take? Let's hit pause for one second. And just for the person listening to

00:47:03.300 this who maybe doesn't know the ins and outs of poker. When we're talking about poker, you're

00:47:07.960 talking about a variant of poker called Texas Hold'em, correct?

00:47:11.080 Not necessarily, but we can.

00:47:12.580 Okay. That's what you played, correct? And that's what most of the pros are playing when we turn TV on

00:47:16.680 and stuff like that?

00:47:17.280 So what you see most of the pros playing when you turn the television on is Texas Hold'em. In terms of what

00:47:22.320 most of them are playing, it's not necessarily that game. That's one of the games that they play.

00:47:27.080 So give us the brief overview of how that game works so that we can understand what those building

00:47:31.580 blocks are that you're talking about, which is everybody within a day could know in detail what

00:47:36.720 you're explaining quickly about how this game works.

00:47:38.880 So this isn't true of all poker games, but it's true of almost all poker games, enough that we can

00:47:45.180 pretend that it's kind of true across the board, that the goal is to make a five card hand. Again,

00:47:50.160 there are some exceptions to this. I just want to make that clear, but I'm capturing most of poker.

00:47:54.180 If I say you want to make a five card hand, those five card hands range from none of the five cards

00:47:59.960 match up. And so all you care about is the top card and the hand. In most poker games, aces are high.

00:48:09.300 In some poker games, they're low, but mostly they're high. Sometimes they're high and low. It depends.

00:48:14.660 So you're trying to make a five card hand. Those five card hands range from very bad, meaning none

00:48:21.120 of the cards match up to each other in any way, to really, really good. And really, really good

00:48:26.980 would be a straight, which would be five cards in rank order. Five, six, seven, eight, nine. That

00:48:34.260 would be a straight. Ten, jack, queen, king, ace. That would be a straight. You can have a flush.

00:48:39.220 That's when all five of the cards are of the same suit. So you have five hearts or five spades or

00:48:43.700 whatever. Full houses where you have three of one rank, king, king, king, and then two of another

00:48:49.600 rank, 10, 10. So your five cards would be king, king, king, 10, 10. So anyway, people can go and

00:48:55.960 look up the hand rankings. The highest rank is a straight flush? It's a royal straight flush. So

00:49:00.280 that would be an ace, king, queen, jack, 10, all of the same suit. All the same suit. Probability of that hand?

00:49:06.720 I'm not sure what it is, but a straight flush is one in 6,000 something. It hardly ever happens.

00:49:13.360 So let's start there. So basically, almost any poker game you play, you're trying to make a five

00:49:17.840 card hand. And there's a very clear set of rules about what the hierarchy and ranking are. There

00:49:22.860 is no ambiguity. There's no ambiguity when you're playing. Now, different games have different

00:49:27.320 rankings because you can play low. So in low, you're actually trying to make sort of what in high

00:49:31.480 would be a bad hand. Okay. So then the other thing about most poker games is for most poker

00:49:37.060 games, you're trying to make a five card hand out of either seven or nine possible cards.

00:49:42.880 That just is also true of most poker games, not all. So in Texas Hold'em, it's a particular form

00:49:50.060 of poker where you have seven cards to work with to try to make your best five. So you're sort of like

00:49:55.720 tossing two of them. But some of the cards, everybody gets to use. They're called community

00:50:03.340 cards. So we can think about the variance of poker. So when you watch something like the Cincinnati kid,

00:50:11.120 for example, Steve McQueen, in that particular game, people are dealt their own five cards. And so

00:50:18.040 you don't get to use any of the five cards that I have. So our cards are unique from each other,

00:50:23.220 and you don't get to see any of my cards. Then you can have variants, which are called stud games,

00:50:29.180 where all of our cards are private, but you get to see some of my cards. So in a stud game,

00:50:35.680 at the end of the hand, I'll have three cards that are face down and four cards that are face up to you.

00:50:42.880 And I'm still making my best five of those seven. And I don't get to use yours. It just gives me more

00:50:48.380 information. It just gives you more information. So once you can see the four cards that are up,

00:50:53.220 gives you some definition to the kinds of hands that I can have. So as an example, because I only

00:50:58.060 have three down, if my four up cards are a spade, a club, a heart, and a diamond, I can't possibly

00:51:06.680 have a flush because the most I could have of any suit now is only four. So you get certain information

00:51:12.780 about the possibilities for what I could be holding because you get to see some of my cards. But yet,

00:51:17.920 you still don't get to use any of my cards. Whereas in Texas Hold'em...

00:51:21.160 Now that's a stud game.

00:51:22.760 But it's common.

00:51:23.760 But it's common. So what happens is you get dealt two cards that are private to you. So these are

00:51:28.140 face down. You don't get to see those particular cards. And then in a particular way, five cards

00:51:34.980 eventually will end up in the middle. So notice that that's seven. But those five cards that are

00:51:39.620 sitting in the middle, both of us get to use. Now, what that means is that these five cards in the

00:51:45.700 middle define for me what your possible holdings are. So that now changes for me what I think that

00:51:53.880 the strength of my hand is. I'll give you a simple example. It just turns out if you have two private

00:51:59.200 cards, the most of a suit that I could have in my hand would be two. So I could have like two hearts,

00:52:04.940 which means that in order for me to ever make a flush, which is five of the same suit, like five

00:52:10.120 hearts, I need to have three more of those out on the board. So if I look on the board and

00:52:15.560 you look on the board, and we both see that there aren't three of a suit that's on the board,

00:52:19.580 it eliminates the possibility of a flush from the other person's hand. That becomes important

00:52:25.000 because if I have a hand that is vulnerable to a flush, like if I have three of a kind or two pair

00:52:32.100 or something like that, that now increases how I think about the strength of my own hand.

00:52:39.820 And you get to look at your cards as soon as they're dealt the two as soon as they're dealt,

00:52:42.920 you get to see your private cards, right? So you know what your private

00:52:45.500 cards are the whole time. And then these community cards are coming down in the middle

00:52:50.020 and those community cards are defining for you what you think the other player might have.

00:52:55.700 Now, this is a lot of where the skill in poker comes from is understanding what those community

00:53:01.600 cards mean. Understanding how they define what the other player has, how they might relate in a

00:53:08.940 logical way to what the other player has, what the probability is that the other player's cards match

00:53:14.160 up in some way to those five cards in the middle. Yeah, it's both, right? It's what do those cards

00:53:19.220 mean about what you have? And what do they not mean about what other players have? Right. And

00:53:24.700 it's interesting because depending on how many players are at the table, and what's the typical

00:53:29.920 number of players that are at the table? And hold them? Yeah. It's generally nine, sometimes it's

00:53:34.680 eight. The higher the limit, they tend to limit the number of players more. So if you're playing in a much

00:53:40.340 lower limit game, you might see 10 people at a table. If you're playing in a much higher limit

00:53:46.000 game, it's more likely to be eight. Which is still kind of staggering to me because now, so you have

00:53:52.420 this math of here's what I have, here's what's common. Based on what's common, what can I impute

00:53:58.800 about everybody else? But now I actually have to pay attention to the behavior of the other people.

00:54:03.560 So I can do the probabilistic calculation broadly. I'm doing two probabilistic calculations. The

00:54:10.680 implications of these cards on me, the implications of these cards on people for whom I don't know the

00:54:15.600 other two cards. But now I have to do eight different behavioral checks. Right. So one thing

00:54:21.340 about the way they relate to your own hand is probabilistic and one isn't. So the thing that

00:54:26.040 isn't probabilistic is these relate to my cards, my hand in a certain way. So if I'm looking at the

00:54:31.160 middle hand, like I know what I have, hopefully. But what is probabilistic, and this is what's

00:54:37.400 really important, is what does that mean for whether I have the best hand or not? So that I

00:54:42.120 don't know, because that's relative to your hand, because there isn't an absolute ranking. It's not

00:54:46.740 like, oh, I have a flush, I get to win automatically. Now, that comes up, like sometimes if I look at the

00:54:54.080 center of the board, I have the very best possible hand that a person could hold. That's very rare that

00:54:59.960 that occurs. Most of the time, I'm somewhere in between the worst possible hand that a person could

00:55:05.900 hold, and the best possible hand that a person can hold. So I have to figure out where I'm sitting on

00:55:10.900 that spectrum, right? And it's probabilistic in nature, whether I'm likely to be beating you or

00:55:15.660 not. And I can think about that in the sense of sort of just base rates, like if I'm holding this

00:55:21.240 strength hand, how often would I be winning? But then I also have to adjust those probabilities for

00:55:27.160 what you're doing and how you're acting. And how you're acting is going to tell me something,

00:55:32.900 two people could act in the exact same way. But depending on my model of who they are,

00:55:37.980 it could mean very different things. So I could bet strongly with a hand. And if you call,

00:55:43.100 that could mean that you have a particular range of hands, that's x. And if another person calls their

00:55:51.420 range of hands that they might call with could be narrower or wider, they could be willing to call with

00:55:56.020 either a hand that's much worse than the bottom end of the range of hands that you'd be willing to

00:55:59.900 call there with. Or they could actually be calling with where their bottom range is better than what

00:56:06.640 your bottom range is. And that's going to depend on that person.

00:56:09.200 As a general rule, does a newcomer to a game have an advantage or a disadvantage? Let's say you and

00:56:14.060 six of your friends have a regular game going, and I show up as a first timer. You guys have never

00:56:20.000 played with me. Does that give me an advantage or disadvantage in terms of the point you just made about

00:56:24.940 ranging me? All things being equal. So let's pretend you know nothing about this. Until we

00:56:29.800 play enough, you don't know if I'm good or not. So let's assume I know the fundamentals of the game.

00:56:34.300 I would say all things being equal against the game, you're at a disadvantage because you don't

00:56:39.920 know how to range six people. But me personally, I would prefer to play in terms of my ability to

00:56:46.460 range you. I'm going to be better with somebody I've already played with before. It depends on what

00:56:50.240 question you're asking. And in fact, there's actually been many, many cases in poker where

00:56:56.240 someone has come in and they're doing something that's just really unusual, that is not something

00:57:05.280 that would be expected. And they do really well a certain period of time. And then the market

00:57:11.600 basically kind of figures out what they're doing. Either they adjust or they don't. But I've seen a lot

00:57:17.000 of players come in and win a lot. In some cases, it's just attributable to their short-term bursts

00:57:22.780 of luck. But in some cases, they're actually doing something really unexpected. And people are

00:57:28.360 actually mis-ranging them. This usually is because they're playing more aggressively with hands that

00:57:34.080 people normally wouldn't play that aggressively with. And then people sort of figure it out and

00:57:38.560 they adjust. And very often that person starts losing after that. Sometimes they adjust and they do

00:57:43.540 other things. But often it has to do with this problem of sort of what do you have?

00:57:48.020 For example, like in cycling, a totally unknown cyclist can win a stage of a race because they

00:57:53.780 lead out, they break out. And the Peloton sort of says, well, there's no way this person can do such

00:57:59.260 and such because we don't know anything about them. And all of a sudden they let them get away.

00:58:03.380 Now they're not going to be able to do that the next day because all of a sudden the Peloton is going

00:58:06.520 to say, well, wait a minute, wait a minute. Now we're going to treat you like we would treat anybody

00:58:09.660 who has the potential to win and your moves now fit into a different frame.

00:58:14.300 Right. Exactly. That's exactly what happens in poker. The first thing that you're trying to do

00:58:18.640 is figure out, I know sort of in an absolute sense what I have, but I don't know where that sits

00:58:24.320 relative to what you have. And then once I sort of figure that out and that's probabilistic in nature,

00:58:29.780 I now have to figure out for you what's the best line of play for me.

00:58:34.680 And tell me how the flow of the game goes from here. So the five cards common,

00:58:38.200 we each have our two. How does the betting work and what changes with the showing of cards or

00:58:44.060 movement of common cards? So you start off, you get dealt two cards and there's a round of betting

00:58:49.660 and you can choose at that point. And this is actually a really important concept in poker.

00:58:54.400 I actually tweeted about a little while ago because I think people should be talking about it more,

00:58:58.200 which is you're totally allowed to fold. Remember when Steve Dubner and Stephen Levitt wrote about

00:59:02.860 this in Freakonomics, like the benefit of quitting. We do not spend enough time encouraging people to

00:59:07.260 quit. No, we spend a lot of time encouraging people to stick to it. And it's actually for

00:59:12.620 everything that you stick to, there's huge opportunity costs involved, right? There's all

00:59:16.580 the other things that you could have done. And if you're sticking to something that you're negative

00:59:20.700 expectancy to, and I'm not saying money, I mean, I'm saying you might be negative expected happiness

00:59:25.160 to it or negative expected health to it or negative expected whatever that, and you're giving up the

00:59:30.820 opportunity to go do something. I mean, I'm a big fan of quit fast. So what I like, my theory is do lots

00:59:37.740 and lots of things that are basically a free roll where there's very little downside to it. Go try out

00:59:42.340 piano, right? It's like a little bit of your time. You can't die doing it. Just go see if you like it,

00:59:48.740 but then quit fast. Just quit fast and move on to something else. And then if you find out,

00:59:54.100 oh, I really love this. I really want to spend my time doing it. Now stick to it. So I'm a huge fan of

00:59:59.100 that. But in poker, this idea is very natural for a poker player because you're doing a lot

01:00:03.320 of the quitting part. You're just saying like, no, this isn't a hand that I mathematically want to

01:00:07.040 play. I don't think I realized that. And that actually might answer this question, which is

01:00:11.680 one of the things I've never understood about gambling in general is you lose much more than

01:00:16.080 you win, presumably. No, not in poker. Ah, because of what you just said. Because you can cut your

01:00:21.420 losses. Okay. That answers the question because my question was going to be, I think it answers my

01:00:24.800 question. Given the power of loss aversion, how the hell do people keep playing? Because you'd think

01:00:32.180 if you only win once out of every 10 hands, that's nine experiences that are each more painful than

01:00:40.100 the pleasure of the win. But of course you don't view those nine as losses. You just view them as

01:00:44.720 non wins. You do lose real hands. I mean, you get to the end of a hand and you lose real money.

01:00:50.160 But a lot of the times, if you really feel like, you know, the table, well, you know,

01:00:53.300 the dynamic, well, you just cut your losses and you're like, I'm going to let these two guys fight

01:00:56.640 it out over here. But right. Exactly. And I'm going to only go to the mat when I really have a shot to

01:01:00.700 win. And presumably losing that still falls into the loss aversion pain. By that point, the pretest

01:01:06.940 probability is such that you have a better shot. So it comes back to this kind of variable ratio

01:01:12.700 reward schedule. You're winning enough big pots. If you're playing this particular form of poker,

01:01:18.040 which is limit, let me just say something about poker comes in two styles. One is limited

01:01:23.120 betting. So in limited betting, it means that, for example, if we're playing, maybe we have to bet

01:01:28.340 in like $10 increment. So I can bet $10. And then if you want to raise the bet, you can make it $20.

01:01:34.040 And then if I want to raise it, I could raise it to 30. In other words, we're raising in $10 increments,

01:01:38.280 and we can't do anything more than that. What comes along with that is there's limited exposure

01:01:42.540 to the person, but it obviously limits what I can win from you on a particular hand. And then there's

01:01:47.960 no limit, which is what you really see on television, which is the super exciting. Someone shoves a

01:01:53.100 million in chips on.

01:01:54.240 Like when I think of poker, I think of rounders. I'm sure there's been a million movies.

01:01:57.580 Which is no limit. No limit and snapping Oreos.

01:02:00.820 I want to be around John Malkovich. I want Oreos. And I want to hear a Russian accent. Like that's

01:02:06.680 all right. Anytime that movie's on anywhere, like if I turn on Netflix, and somehow it pops up,

01:02:11.180 I'm watching it.

01:02:12.340 Right? Yeah, that's no limit, which is like that super sexy.

01:02:15.380 Oh my God, it's incredible.

01:02:17.160 I'm going to lay everything on the table. Can I bet my lung? But there's also something called

01:02:23.280 limit. So in limit, a really good limit player at the end of an eight hour session, a very good limit

01:02:31.680 player is going to have won about 56% of the time. This is a really, really excellent player. Now go talk

01:02:38.120 to anybody on Wall Street. They can get a hundred billion AUM if they say for every bet has a

01:02:45.360 56% chance of winning, right? I mean, this is a huge ad. So I'm saying this is a really,

01:02:49.740 really, really good limit player. What that means though, is that the person that they're

01:02:53.400 playing against is going to win 44% of the time. So think about how much reinforcement they're

01:02:58.180 getting. So sometimes they're going to win three sessions in a row, right? I mean, 44%,

01:03:01.960 that's a lot. This is a great player.

01:03:04.740 That's hard to believe, even when you bring it back to the context of what we keep using as a

01:03:08.240 great comparison, which is sports. Think about this. If the New England Patriots played,

01:03:12.740 let's see, who was the worst team in the league last year? I don't even remember. If you put the

01:03:16.680 best team against the worst team, it would not be 56, 44. It wouldn't even be close. It would be

01:03:21.860 85, 15 or 90, 10, even in professional sports. So that tells you how much narrower the gap is here

01:03:29.780 in poker. This is specifically in this limit version of poker. Now in no limit, you can get these bigger

01:03:35.220 spreads. Ah, the 56% is dollars, not number of times in which a person wins.

01:03:42.020 56% of the time is at the end of an eight hour session. How many times have I won?

01:03:46.540 Have you won? Okay.

01:03:47.500 At the end of that session. So over eight hours. Now, just to allow you to understand,

01:03:52.920 it's just a matter of the number of iterations because it's sort of in the same way that if

01:03:57.400 the Patriots are 56% against another team, if they play enough times, they're 100% to have won the

01:04:04.400 series. So a poker player who's going to win 56% of the time at the end of eight hours, at the end of

01:04:09.560 1500 hours will approach to 100% that they'll be in the money at the end of the year. And this is

01:04:14.520 in this limit version of poker. Now, when we get to no limit, basically what's happening is that

01:04:19.120 you're realizing, you're more likely to realize you're... You're going to see huge volatility and

01:04:24.540 you have enormously asymmetric outcomes at times. Right. So here's an interesting thing,

01:04:28.860 actually, because of that, is that when I started playing poker... I love that we haven't even got to

01:04:33.640 you playing poker yet, but that's okay because we're going to get there. When I started playing

01:04:36.760 poker, the pool of players was small. So it wasn't on TV. It wasn't on the internet. There just wasn't

01:04:44.460 this huge market of people who were like, oh, I'm going to go and play poker. So what that meant was

01:04:51.240 that this limit version of poker was really what was being played in casinos by professionals.

01:05:02.160 And the reason was that when you had someone come in who was not such a great player, making sure

01:05:09.080 they were getting that frequent reinforcement was really, really, really important. So you can think

01:05:14.980 about it this way. If you have kind of an unlimited supply of new players coming into the game, when you

01:05:20.360 come in, I should just take all your money right then because who cares if you get depressed and

01:05:24.760 go away? Who cares if I hit you over the head and it plays more like chess and you're like, well, 0.50

01:05:32.020 I'm not going to play anymore because... It's not even close. It's not even close. That's ridiculous

01:05:35.860 because there's literally 17 people standing behind you waiting to take your seat. So now what I can do 0.74

01:05:41.940 is I can just maximize in that moment. I can just take as much of your money as quickly as possible

01:05:46.120 because there's going to be somebody else willing to come into the game. When I first started

01:05:49.660 playing, that was not true. When I first started playing, if someone came in the game who was

01:05:53.460 really playing at negative expected value, you didn't want to take a sledgehammer to them.

01:05:59.000 You wanted to make sure that they were enjoying themselves, by the way. And so playing these

01:06:04.440 limit games, which were kind of like a softer way that the edge played out over a longer period of

01:06:09.800 time, was kind of the way to go because then they won enough. It was like they won enough of the

01:06:15.740 time to feel pretty good. And the thing is that I think that people focus when they're thinking about

01:06:21.380 poker on the expected value being about money. And when this person came in and they were losing

01:06:28.560 money, it wasn't that awful for them. When somebody goes to see a sports game, they play money. They get

01:06:33.760 something in return for it, right? They get to watch a good game. When somebody goes to a restaurant

01:06:39.120 and they order food, they're giving money.

01:06:43.380 Yeah, there's a guarantee expected loss.

01:06:45.180 Right. But for them, what they're getting in return is of value. So generally, when these people

01:06:50.860 would come into the game who would be playing at negative expected value, if you only thought about

01:06:54.680 money, I think that most of them were actually playing at positive expectancy when you looked

01:06:59.400 overall at what their overall experience was. Now, the thing is that no matter who you are, it's not fun

01:07:04.440 for somebody to hit you over the head with a sledgehammer. Like here, just give me all your money.

01:07:08.260 It happened in two seconds. Now go away. But when they would come in and they'd get to play with

01:07:14.460 really good players and they'd be learning the game, which is exciting. And they'd feel like,

01:07:19.260 you know, I'm playing at table one at the Bellagio or Aria or whatever. And they're winning enough

01:07:25.420 to feel good. They're probably learning. And this is entertainment for them. Now it's no longer zero sum

01:07:33.420 anymore. It's like, is it zero sum as far as the money is concerned? Sure. But it's not

01:07:38.240 zero sum overall when you look at like, what's the scope of what people value, because there's all

01:07:42.540 this entertainment happening as well. And I remember there was this guy, I won't say his last name,

01:07:46.780 but this guy named Jay, who came and played, he came in the late 90s. And he, you know, he's probably

01:07:52.320 mid to late 70s. He had made a bunch of money on Wall Street, moved to Las Vegas to retire. His health was

01:08:01.440 okay. His mind was super sharp. And I think that he just sort of decided this is what he wanted to do

01:08:08.300 for his retirement. He had a ton of money. And so he started playing in these high limit games with

01:08:13.360 the high limit players at that time. And it was a small group. And he'd come in and play. And the

01:08:19.840 whole group would break for dinner. And we'd all go with Jay to go eat at a restaurant in the Mirage.

01:08:26.740 It was at this time. This was before the Bellagia was built. We'd all go eat dinner. And he would

01:08:31.240 sometimes have parties at his house. And we'd go over to the party at his house, or he'd get invited

01:08:35.380 to somebody else's house. And this was his social life. And he was losing a lot of money, but he was

01:08:40.800 happy. He was having fun. He had this whole group of people. And he got to have this fun conversation.

01:08:46.960 And he got to learn this game and so on and so forth. And I think this is kind of how he spent the last

01:08:51.740 four or five years of his life. And he was for sure winning to that. So I feel like people sort

01:08:57.420 of think it all sounds very cutthroat. But it's not. And this is something that certainly I think

01:09:02.780 that the players in the 90s, for sure, were really thinking about, which is what's the value add? How

01:09:08.380 are you bringing value for this? We know how they're bringing value to us.

01:09:12.420 Wow, that's so interesting. I wouldn't have guessed that. But that's a nice symbiosis.

01:09:15.440 There's a really, really famous player named Chip Reese. He sadly passed away about,

01:09:20.880 very unexpectedly, he was young. Passed away probably about 15 years ago now. Maybe the

01:09:25.760 greatest poker mind ever. I mean, certainly one of them. And the thing about Chip was he just made

01:09:31.620 everybody happy around him. He made a ton of money playing backgammon. He made lots of money playing

01:09:37.280 poker. And he had the superpower of these people who were very negative expectancy, if you were thinking

01:09:42.820 about money, would be calling him up like dying to play with him. Because he was nice. It was just

01:09:48.020 a really good time. I think about it as the same thing. Why do you go to a movie? Well, okay, if we

01:09:53.400 just think about it as money, it seems like a losing proposition. But of course, that's not what we should

01:09:58.520 be really thinking about that as. And so I think that that gets lost a little bit as the game grows and

01:10:04.020 becomes really, really, really big. And so if you don't have a particularly good time, and you're sad,

01:10:11.140 and you leave, there's somebody else just to take your place. And I think that that interesting

01:10:17.460 aspect of how it really wasn't very zero sum at all kind of goes away. I think it's come back a

01:10:23.960 little bit now because the poker economy has contracted. And I think they're sort of coming

01:10:28.680 back to that idea of how does this become fun for everybody? It's a really interesting aspect of the

01:10:33.840 game back then. So when did you figure out you had a knack for this? It's hard to say because I feel

01:10:38.200 like it's all relative. You know, I retired in 2012. And pretty much anything else that I can

01:10:44.320 think of that's competitive, poker's really changed the ability for people to be able to

01:10:50.600 run batches of hands, for example, after the fact is sort of think, oh, here is the situation that I

01:10:56.200 was in. Now I'm going to go and I'm going to run this and sort of see what the right answer would

01:11:00.300 have been so that I can take that into the next time I play. The kind of analytics that people have,

01:11:05.040 the ability for them to discover what the equilibrium point is on a particular decision

01:11:10.360 or whatever. It's just beyond what I can even comprehend. So I left before I would have had to

01:11:16.880 put that work in. If I did put the work in under those conditions, I'm not sure that I would

01:11:22.260 do well. I mean, I know that if I sat down today without doing that work, I would get completely

01:11:26.800 crushed. So in that sense, you can look at me and say, well, I didn't have a knack for it. If I tried

01:11:31.920 to play against today's players who know the things that they know and are able to run the

01:11:36.960 kinds of simulations and analytics that they can run now and they're understanding the mathematics

01:11:42.880 of the game are and sort of what the optionality is. So you're saying that a professional player

01:11:47.500 today will take every hand of a game and they'll have the simulation after the fact of that hand?

01:11:53.240 Well, not necessarily every hand, but they can pick hands and they can actually run the same way

01:11:57.440 that you do it in baseball or... Well, I was going to say, this sounds a lot like Moneyball,

01:12:00.780 which is the advent of analytics to the game. Right, exactly. So they can sit there and they

01:12:05.660 can say, they can just set the parameters. If I am in a particular situation where I think that I have

01:12:11.500 to bluff and if I give you this range of hands, how often do you fold? If I give you this range of

01:12:16.660 hands, how often do you fold? And they can figure out exactly at what point, if I bet a particular

01:12:23.160 amount, am I winning enough to that bet that I can now try to bluff in that situation that maybe

01:12:28.740 I would have never thought in a million years was a situation that I would be winning to if I were

01:12:32.740 to bluff there because I just, I didn't have the data. So I feel like knack is such a strange word

01:12:39.300 because I feel like it's knack relative to what? Well, to your peers at the time. Yeah. So those peers

01:12:45.760 changed. So I started playing in Montana. What year is this? Oh my gosh. I started playing,

01:12:50.600 I turned professional in 94. So it's right around that time. And I was playing against people in

01:12:55.740 Montana who a lot of them were retirees who were, this is kind of what they did. Some people have

01:13:01.920 their bridge game or some people are playing golf or some people are playing poker and that's kind of

01:13:07.000 what they wanted to do. So while I think that pretty much everybody who plays poker would like to get

01:13:11.680 better and enjoys the game more, some people are more hungry to do that than others. And people are

01:13:18.920 getting different things out of the game and the people that I was playing with that were not

01:13:23.960 thinking, let me become a world-class player. At that time, particularly because I had really

01:13:30.200 great mentors, my understanding of the mathematics of that game was so beyond their understanding. I

01:13:36.260 don't think that was part of their enjoyment. So the day that I stepped into that room, I had a knack

01:13:42.500 certainly compared to the people that I was playing against. I started playing two years before that in

01:13:47.580 that game and that was sort of where I was learning. And then in 94, I decided I was a professional

01:13:52.740 poker player and I moved to Las Vegas and I started playing there. And this is how many years after the

01:13:57.040 game in New York that, or was it only your brother that was playing? Only my brother was playing. Yeah,

01:14:01.240 I never played in New York. I don't think I ever played a hand of poker in New York. So you get to

01:14:05.440 Vegas in 96? No, I got to Vegas in 94. Oh, that was the declaration of running pro. Okay, got it.

01:14:11.100 Does it become less fun when it's your job now? Does it amplify the stakes? Eventually, but no,

01:14:16.340 not at this time. Poker is a really complicated game. Very, very complex. And I know that there's

01:14:24.140 just so much that I don't know about the game. And I'm totally aware that if I tried to sit down and

01:14:29.320 play against today's players, that they would bury me because there's this whole layer of the game that

01:14:35.000 I haven't discovered. But while I was particularly in those early years in the 90s, I had started off

01:14:42.840 in Montana playing only limit Hold'em. Now I go and I start to learn more about that game. So I'm

01:14:49.200 starting to sort of peel back the layers of that game. And like most things that you do, as you peel

01:14:54.400 those layers back, what you start to see is that there was a whole bunch of stuff underneath it that

01:14:58.020 you didn't even know existed. Right? So you think that the onion is a certain size, and you peel a

01:15:03.340 layer back and you realize, oh, wait, no, there's like so many more layers under there than I ever could

01:15:07.380 have thought. I was telling this to somebody the other day, my head of research always says,

01:15:11.120 I don't know if he's paraphrasing somebody or if he coined the term, but the further you get from

01:15:15.100 shore, the deeper the water. Somebody asked me a question about a topic. I don't remember what it

01:15:19.060 was. But I said, and I wasn't being facetious, I really mean it. I know less about this subject

01:15:24.720 today than I did 10 years ago. And they said, how is that possible? And I said, well, to be completely

01:15:29.560 accurate, I mean, on a relative basis, obviously, on an absolute basis, I know more than I did 10 years

01:15:34.160 ago about subject why. But my knowledge of how much broader it is today has dwarfed my absolute

01:15:42.220 knowledge. So on a relative basis, I've gone backwards. Yeah, exactly. That's what's happening.

01:15:47.760 So I'm learning this game limit hold them, learning things about poker in general. And then I start

01:15:53.260 expanding my repertoire of the types of games that I play. So I start to learn the Omaha games,

01:15:58.120 I start to learn the stud games, I start to learn pot limit and no limit. So those are different

01:16:03.040 betting structures. And I'm immersed. It's like I'm living and breathing and talking poker

01:16:08.720 all the time with peers, with mentors, so on and so forth. So it's not really like a job in that

01:16:17.640 sense. Certainly not during that first eight or 10 years that I'm playing, I'm learning. And every

01:16:25.760 second is learning. So when did you see sort of robots in the book, but you also figure out

01:16:30.900 it seems pretty early that there are at the risk of oversimplifying two different types of players

01:16:36.600 through the players who, when they win, it's because of how well they've done things when they lose,

01:16:43.620 it's because of bad luck. And that's basically the narrative. And you seem, and again, I don't know

01:16:50.400 how quickly you came to this understanding. But at some point, you seem to come to the understanding

01:16:54.960 of, wait a minute, sometimes I'm winning when I make the wrong bet. And sometimes I'm losing when

01:17:01.680 I make the right bet. And that last one, by the way, must be a hard pill to swallow.

01:17:05.800 You end up sort of in your own mind formulating a very common tool we use today, which is this

01:17:11.180 decision matrix of, did I use the right process? Yes or no. Did I get the right outcome? Yes or no.

01:17:18.820 That's a lovely two by two, but it's one that you have two squares of that two by two never get

01:17:24.820 talked about, which are the ones that are at odds with each other. Let me really dig into something

01:17:30.060 you just said. So I agree that we're not seeing the whole two by two. Where I'm going to quibble

01:17:36.060 is this. Correct me if I'm wrong, but I think what you said was you could have made really good

01:17:42.160 decisions and had a good situation and ended up having a bad outcome. And that's the really hard

01:17:47.900 pill to swallow. And I actually think it's the reverse to me. I think that's the story you tell

01:17:53.740 when you're opening your book, which bringing it back to the England Patriots is when the tables are

01:17:57.900 turned, right? It's the Pete Carroll pass call on the two yard line or whatever in the Superbowl that

01:18:05.240 gets intercepted, which again, by all metrics, if you looked at this, like you were a robot looking at

01:18:11.760 reams of data, the outcome of that call was positive. That should have been a touchdown.

01:18:17.900 Right. That's a question of resulting. When we're evaluating other people's decisions,

01:18:22.500 particularly people that we don't compare to, particularly in a situation where the decision

01:18:26.440 is complicated, we have this simplifier, the shortcut called resulting. So what we do is we say,

01:18:34.240 if I know what the quality of the outcome is, I can work backwards from that to the quality of the

01:18:39.120 decision. So this is what happens to Pete Carroll. It's the last play of Superbowl 2015.

01:18:44.060 Everybody expects he's down by four to the Patriots. As always, it's always the Patriots

01:18:50.340 back to the beginning of the conversation. At any rate, everybody thinks he's going to hand the

01:18:54.000 ball off to Marshawn Lynch. Instead, he has Russell Wilson, the quarterback pass the ball.

01:18:58.460 This is very unexpected. The math is very, very complicated here. There's some game theory in

01:19:04.480 there. There's options theory, all sorts of things that people are interested in reading about it.

01:19:08.400 They can go look at Benjamin Morris on FiveThirtyEight. Suffice it to say, it is a good play.

01:19:12.800 We'll link to all of this in our show notes, by the way, because we love nerding out on this exact

01:19:16.480 sort of background. So he passes the ball. It's intercepted. They lose the Superbowl this way.

01:19:23.160 Malcolm Butler, my hero. Just, I mean. I can't imagine. Can you imagine what I was doing to my TV?

01:19:30.260 I remember looking at my daughter going, earmuffs. And it was like, fuck, yeah. I mean, I am screaming. 1.00

01:19:40.000 My wife's like, all right, I'm going to take the kids out. I'm like, yeah, yeah, yeah, that's fine.

01:19:42.900 Yeah, exactly. So this is what we do. The thing about understanding the quality of the decision,

01:19:47.720 it's complex. It's really hard. You need to know a whole bunch of different things. You need to know,

01:19:54.000 given the decision that was made, what were the possible outcomes? How often would those outcomes

01:20:00.760 occur? If you think about what are the consequences of each of those outcomes? How much do I prefer

01:20:05.740 each of those outcomes? So I can understand what the probability of getting a preferred outcome is.

01:20:09.900 If I have a outcome that I don't prefer, like an interception, like how often does that happen? And

01:20:14.660 then you also have to compare it, do that exact process with the other possibilities, the other

01:20:20.060 decisions that you could have made, and now actually compare to see how do I get the highest

01:20:24.340 expected value, the highest chance of actually winning the Super Bowl. And it's all very complicated

01:20:28.920 and involves what's the interception rate, which people don't know off the top of their head. Or

01:20:34.580 you have to think about if there's only 26 seconds left, how do I get three plays off instead of two?

01:20:39.740 And it's very hard. Suffice it to say, if you pass the ball either on the first or the second play

01:20:45.840 there, because it's second down, 26 seconds left, one time out. If you pass the ball on either the

01:20:51.160 first or second play, it means that you can do that plus hand the ball off to Marshawn Lynch.

01:20:58.120 If all you do is hand the ball off to Marshawn Lynch twice, you don't get that third play.

01:21:02.580 That third play is incredibly valuable. And the cost of the third play is whatever the

01:21:07.220 interception rate is. And the interception rate there is less than 2%.

01:21:10.380 And there's a time management piece to it as well. A pass that is not caught is an immediate time

01:21:16.500 stop. You could have a bizarre scenario under which you hand it off to Marshawn Lynch.

01:21:21.340 It turns into a bit of a scrum. You have to burn one of your timeouts.

01:21:24.820 Well, you're going to have to no matter what.

01:21:26.320 You only have one left, if I recall.

01:21:27.960 Right. So you have one left. So that's why if you hand it off to Marshawn Lynch

01:21:32.540 on the first two plays, assuming the first one fails, that's the end of your timeout. So

01:21:37.940 because you can get the clock to naturally stop on a pass, that's why if you pass on the first or

01:21:43.620 second play, you can get three plays instead of two. But that's all very complex. So what we tend

01:21:49.060 to do in those situations is we say, that's all really hard. And I don't know what any of that is.

01:21:54.800 And it's all very opaque to me. So if I understand what the quality of the outcome is, was it good or

01:22:00.880 bad in this particular case, it was disastrous. Then I know what the quality of the decision is. So

01:22:05.740 that's what's happening in that Pete Carroll thing. That's called resulting. Resulting is a heuristic

01:22:10.060 simplifier. Works really well in chess. If I lose to you in chess, I played worse than you. Works very

01:22:16.340 poorly in poker. If I lose to you in poker, who knows? And certainly very poorly in football. We

01:22:22.700 don't really know, depending on how the play worked out, whether it was a good or bad decision. But what's

01:22:28.040 interesting is that people don't do this resulting thing in situations where they feel like the quality

01:22:35.460 of the decision is known. And I say feel like, because- In quotes, yeah. Right. Because they

01:22:40.660 don't necessarily know it. But remember that what I said is resulting is a simplifier. The quality of

01:22:45.700 the decision, it's complicated. It's hard to figure out. So what happens when we feel like we do know the

01:22:49.980 quality of the decision. So I'll give you an example. If I come home one day and I say to you,

01:22:57.900 I got in a car accident. Do you know if it's my fault or not? No clue. And you won't use the fact

01:23:04.900 that I got in a car accident to work backwards like you do with Pete Carroll, because you understand

01:23:11.160 that there are certain things that really are knowable about driving. And you can ask me what those

01:23:16.060 are. Did you break the rules of the road? Were you drinking? You can ask me a whole bunch of things.

01:23:22.240 So there's all sorts of places like that where we feel like we understand what the decision quality

01:23:27.380 is. And so if we have a bad outcome, we actually tolerate it. This occurs all the time in business

01:23:32.440 as well. Right. If somebody makes a decision, let's say they execute on a decision that was made

01:23:38.100 in a committee where the whole team agreed that this was the right way to go. And then it doesn't

01:23:47.200 work out. People aren't like idiot. But if they do something on their own, that is not something that 1.00

01:23:55.520 everybody always does, then all of a sudden, if it doesn't work out, they're really under the gun.

01:24:01.180 So we have this issue of transparency that occurs. So now let me get back to this roundabout way.

01:24:07.600 So first of all, let me just give you a good example of the transparency problem.

01:24:11.280 Okay. Bear with me on this thought experiment for a second. You don't have ways. It's broken.

01:24:17.340 And you're driving to the airport with your spouse. Obviously, you have to get there in time. And you get

01:24:25.060 in the car and you go the normal way that you go, that you've always gone. And something happens,

01:24:31.400 there's a big accident on the road, and your car is not moving. And you get to the airport,

01:24:37.220 and you just missed your flight. Are you getting blamed for that? Is that your fault?

01:24:41.200 Is your spouse mad at you?

01:24:42.640 No, not in my case. I can imagine in some people's cases.

01:24:45.140 But mostly, no, it doesn't feel like it. Okay, so now you get in your car. Again,

01:24:48.620 you don't have ways, whatever your phone's broken. You get in your car,

01:24:51.640 and you announce to your spouse, I have a shortcut.

01:24:53.880 But I was just talking to Morgan over there, and they told me about this great shortcut. So I'm

01:25:00.980 going to take the shortcut because I've checked it out. I looked on a map. It's better. And now

01:25:05.540 you go to the airport, and there is an accident. Car is not moving. You make it to the airport,

01:25:10.660 and you've just missed your flight. Is your spouse mad at you?

01:25:14.420 Again, mine would not be because she's superhuman. But yes, I think many people in that situation are

01:25:20.320 absolutely getting yelled at. By the way, that's a superhuman spouse. It's like, 1.00

01:25:23.960 I cannot believe your stupid shortcut. So this is one of those cases where, 1.00

01:25:28.220 so we have this come up all the time. And when it comes up in so many different ways,

01:25:33.640 it comes up in medicine all the time. Because that's such a great example, right? If Marshawn

01:25:38.080 Lynch had been handed the ball, and he had fumbled the ball, same exact outcome. People would be pissed

01:25:44.260 at Lynch. Nobody would be pissed at Pete Carroll. That is exactly right. And they might not even be

01:25:48.620 pissed at Marshawn Lynch. They might just think that the Patriots line is too good.

01:25:52.940 Yeah, the safety, or whoever came and stripped the ball out was just exceptional.

01:25:56.760 Right. And why is that? What's the difference? Because in both cases, you fail.

01:26:00.100 Expected versus not expected.

01:26:01.700 Expected versus not expected. And then this comes up, I guarantee you this comes up in your business

01:26:05.580 all the time. It comes up in your personal life all the time. It's one of the reasons why,

01:26:10.920 for example, if someone's really unhappy in their job, they won't be aggressive enough about going and

01:26:17.200 finding another position. And the reason is that the unhappiness in their job is now sort of become

01:26:23.440 expected. Like they're not really blaming themselves for that. But if they go and find a

01:26:26.820 new position, and they're not happy there, they're going to feel like it's their fault. As an example,

01:26:30.860 this happens in business strategy. This is the way that teams end up operating, so on and so forth.

01:26:35.800 Okay, so this becomes a really big problem. So every human being understands this difference between

01:26:40.960 kind of transparent and opaque decisions. Now, here's my question for you. What decisions do you

01:26:46.120 think are the most transparent to you? Your own. These have the most transparency to us.

01:26:53.120 So what happens with our own decisions is that we allow uncertainty to bubble to the surface as the

01:27:01.220 explanation for a bad outcome more often than we would for somebody else's decisions. Why? Because

01:27:07.540 we feel like we know what the quality of our decision is, number one. And number two, we don't

01:27:12.040 really want to question the quality of our decision. So you can think about that sort of in general,

01:27:16.340 do you hand it off to Marshawn Lynch or do you pass it, is a question of when are we allowing

01:27:20.520 uncertainty as the explanation for a poor outcome. So it's innovative, unexpected, opaque, all of those

01:27:28.220 things. We don't allow the uncertainty in the door. But when we feel like we know, because obviously

01:27:33.140 you're supposed to hand it off to Marshawn Lynch, now we let uncertainty into the door. We do this for

01:27:37.280 ourselves as well. So now this brings me full circle to this thing about when we think about

01:27:42.240 this matrix and you say, well, you might make a really good decision and have a bad outcome.

01:27:48.960 That's the most painful, you said. By the way, that's just my subjective overlay. I don't know

01:27:54.280 that there's any decision-making expert out there that would argue one way or the other.

01:27:58.080 Well, I think that people do think that. So I just want to sort of pick at something. So let's just

01:28:01.920 start again. So what you said was there are these two quadrants, right?

01:28:05.540 That we would think of as discordant.

01:28:07.260 Right. So one of them is you make a bad decision.

01:28:10.960 But you get the right outcome.

01:28:12.220 You get the right outcome. And then you said even more painful is you make a good decision

01:28:16.660 and you get a bad outcome. Now, what I think is interesting is that I would actually argue

01:28:21.360 that that doesn't have a lot of pain associated with it. And let me explain why. Imagine you're

01:28:27.820 looking at this in the aftermath of the outcome. Okay. So in one case, you've gotten a good outcome.

01:28:32.200 In one case, you've gotten a bad outcome. So let me ask you to do the thought experiment.

01:28:36.320 Where do you want to dig in more? If you get a good outcome, are you looking to dig in to try

01:28:42.200 to find out if you made a bad decision that resulted in that good outcome? Or are you just

01:28:46.160 being like, good outcome? Yay me. But if you have a bad outcome, are you eager?

01:28:51.340 Honestly, I think it depends. Like using sports as an example, I think is a different example

01:28:56.660 versus in medicine. So I've talked about this before on the podcast. In medicine, there's a,

01:29:01.700 or at least in surgery, there's a conference that's typically done called the morbidity and

01:29:05.720 mortality conference, where you, as the name suggests, every week you discuss all the morbidities

01:29:12.160 and all the mortalities of the previous week. And so that is an objective definition. A person

01:29:19.300 can basically arrive dead in the hospital from, let's say a gunshot wound. They still get presented

01:29:26.260 in the conference, even though, let's say you did nothing. They literally showed up in the emergency

01:29:31.200 room. You did CPR for a minute. They were all basically already dead in the call time of death.

01:29:34.560 That still shows up, even though there's no blame to be assigned within the confines of how

01:29:38.860 medicine was practiced. So those ones don't get a lot of discussion. The ones that get a lot of

01:29:43.160 discussion are you made a decision that seemed like the right decision at the time, and it seems

01:29:50.060 defensible, but the outcome was bad. Now, of course, as you've alluded to, these are so complicated

01:29:56.100 because there's still the luck component. There's all these other things that go into it. So I guess

01:30:00.480 if your question is, where do you learn the most versus what causes the most consternation? Those

01:30:06.800 aren't necessarily the same, are they? The point that I'm making is not so much about where do you learn

01:30:10.580 the most? Because I think you learn from all four quadrants, and it's really important to learn

01:30:15.500 from all four quadrants, right? I made a good decision. I got a good outcome. I'd like to

01:30:18.720 understand when I have made a good decision that has a really good expected value, and I want to be

01:30:25.260 able to see that. I want to understand when I made a bad decision that had a bad outcome and where I had

01:30:29.780 negative expected value, and then I can understand that so I can adjust my decision making going

01:30:33.640 forward. And likewise, I'd like to understand when I made a bad decision that had a good outcome and vice

01:30:37.520 versa. What you just described is actually the whole shebag in terms of the reinforcement of this 0.98

01:30:44.380 bad kind of thinking. Because what I'm guessing does not occur every week is the, wow, this patient

01:30:50.780 did way better than expected. But we made the wrong decision. Let's have a conference about that. So let

01:30:56.240 me give you a super simple example to try to get at this, because I think that this is actually a really

01:31:00.360 big problem, particularly as I feel like now people have now understood the importance of process

01:31:08.960 language. So if we think about any business movie from the 1980s, they're like, I'm results driven,

01:31:16.320 right? Now, now you sort of look back at that, and you're a little horrified, because I think we all

01:31:20.180 understand that we don't want to be results oriented. We kind of get that that's a problem. And we'd all

01:31:25.160 like to aspire to be Sam Hinckley and trust the process. And we know we're supposed to say process,

01:31:29.500 process, process, process. But here's the problem. So now let's do this as a two by two. Okay, you have

01:31:35.820 a status quo decision, the expected decision, a consensus decision, and you have a good outcome.

01:31:45.480 Is anybody hailing you as the biggest genius of all time? No, but they're saying good job. You have a

01:31:52.040 status quo decision that has a bad outcome. Is anybody excoriating you? No,

01:31:59.080 they're saying tough luck. What could you do? You have an unexpected, an opaque, a non status quo,

01:32:07.080 a non consensus decision that has a good outcome. And they're hailing you as the biggest genius of

01:32:14.080 all time. You have a non status quo decision that has a bad outcome. And everybody's calling you an

01:32:20.660 idiot. So people understand this. So now let's think about what does that do to somebody's decision 1.00

01:32:26.620 making? Well, first of all, they want to avoid the bad outcome. This is where loss aversion comes in,

01:32:32.220 right? We don't want to be the idiot. That's the one quadrant that you want to avoid, right? Well, in 1.00

01:32:36.440 general, we want to avoid the whole right side of the table because the right side of the table contains 0.99

01:32:42.280 the idiot section. So let's think about how can we avoid the whole right side of the table? Well, one way 0.89

01:32:48.620 we can avoid sort of the bad outcomes altogether is to kind of fall down on the not deciding thing,

01:32:55.960 just sort of not doing anything that's going to cause a big loss, or a big win. Either you could do

01:33:03.480 it by omission, you could sort of not act, you could make very, very low volatility choices, there's

01:33:09.440 things that you could do there to play these strategies. The other thing that you can do, so you can

01:33:14.820 avoid deciding at all, or you can make sure that you're making things that don't really create big

01:33:20.540 losses, so you're not going to end up in the room. But the other way to deal with that whole right side

01:33:25.920 is to focus on the lower right quadrant and say, I never want to be the idiot in the room. And so 1.00

01:33:31.380 therefore, I'm going to make status quo decisions. So let's think about this. For example, it doesn't

01:33:37.460 matter if 80% or so of ear infections are viral. If a patient comes into my office, I'm giving them

01:33:43.740 antibiotics because I don't want to get yelled at. If this happens to be the time, I want to be able

01:33:48.720 to cover my butt. And that way nobody can yell at me. So now let's think about this. So what ends up 0.96

01:33:54.920 happening is that we want people to be thinking, I just want to make the best decisions. And I don't

01:34:01.320 really want to be afraid of the bad outcomes. I particularly don't want people to be afraid of

01:34:06.980 this idiot quadrant, because we would like people to be innovative. And we'd like them to be thinking 0.99

01:34:12.080 outside the box. And we'd like them to sort of push against what the status quo is, because that's how

01:34:16.140 obviously we move forward as a society, as a business, as an individual. But this is what

01:34:22.740 happens. Everybody has a morbidity conference. So even if in the morbidity conference, you're talking

01:34:29.300 about process, you're in the room because of morbidity. So everybody knows I get in the room because of

01:34:35.820 something bad. So here's a simple thought experiment that I use. Imagine that we're investing in

01:34:41.800 real estate, like we have a real estate investing group. And we have a model of the market, obviously.

01:34:46.720 And we have limited resources that we can deploy. And so we invest in a particular project, and it ends

01:34:54.500 up doing 15% worse than we expected. We're all in a room. Now, we might be in that room talking about

01:35:02.300 our model. True. We might be in the room talking about our decision process. But we're having a long

01:35:09.220 meeting in a room about the fact that the project did 15% worse than we expected. Now we do the exact

01:35:15.480 same thing. We invest in this particular project, and it does 15% better than expected. Is the same

01:35:20.980 conversation happening? No. So I don't care how much the word process comes out of your mouth.

01:35:26.980 Everybody knows they're supposed to be afraid of bad outcomes then. And here's the really big problem.

01:35:32.420 And it's true, we can talk about it in terms of medicine as well. But this particular thought

01:35:36.300 experiment that has to do with the real estate, it matters just as much as if you've overestimated

01:35:41.360 the market, which is how you'd end up 15% on the downside, than if you've underestimated the market.

01:35:47.200 Because your goal is to efficiently allocate your resources within the market that you're

01:35:53.140 allocating the resources in. And if your model's wrong, I don't care if it's wrong, if it's

01:35:58.460 underestimating or overestimating, that's a problem for efficient allocation of resources.

01:36:02.360 Not only that, but when it's 15% to the good, it's a signal that there may be risk that you

01:36:08.780 didn't identify. So for example, you could have the mean right, but you may have the variant.

01:36:14.820 That's a really big problem, because that has really big implications for what percentage of

01:36:19.500 your bankroll, or of the total money that you have, the total capital that you have that you're

01:36:24.100 supposed to deploy to that particular project.

01:36:26.500 That's a very important point, by the way, which doesn't, I think, get talked about enough,

01:36:29.800 even in that circle, which is people are very fixated on return on capital,

01:36:34.480 but they generally aren't as appreciative of risk-adjusted return on capital. So Rayrock

01:36:39.360 versus Rock is this concept that seems to be missing from the way a lot of people think about that.

01:36:44.960 So it's good that you brought that point up, because there could be a number of problems with

01:36:49.080 that issue, not just maybe you didn't allocate enough, maybe you allocated the right amount or too

01:36:54.120 much, it didn't appreciate what was at risk.

01:36:55.820 Right, exactly. So we want to be seeing those signals, right? Because we'd like to understand

01:37:00.400 it if we underestimated the vol. And in both cases, your model could be totally right. And you can just

01:37:05.400 be a couple of standard deviations away, or there's a variety of reasons on that particular time that

01:37:11.060 it could have come in low or high, that don't have anything to do with your model. But you kind of

01:37:15.460 want to dig in either way, because it's something unexpected happened. The other thing, by the way,

01:37:19.920 that happens is that when you have a downside event, and you do get in the room, people rarely

01:37:25.800 asked, should we have done worse? They asked, could we have done better? And now again, you're

01:37:30.380 reinforcing for people, I care when things go badly, when things go badly, I want to avoid

01:37:35.640 them. Now poker happens to be this very pure place, where should I have done worse, regardless

01:37:42.260 of whether I won or lost is actually a very important question. Because sometimes you have

01:37:47.720 a hand, I'm thinking about what you have. And let's say that I put you in a range where you're

01:37:53.300 very weak. And I feel that I'm very strong in comparison to you. So because you're weak,

01:37:58.420 I'm playing the hand, we would call it slow or small, where I'm...

01:38:03.140 You're trying to draw as much out as possible.

01:38:04.760 Right, which means that I'm not betting really big. Now at the end of the hand, you hit a lucky

01:38:08.280 card at the end of the hand, the very last card, I end up losing the hand. But when we turn the

01:38:12.820 hands over, it turns out that you were much stronger than I thought. In this case, I should have

01:38:18.260 lost more. Because yes, you hit a lucky card, but I should have been extracting much more money from

01:38:24.700 you because you would have tolerated it had I had your hand right. Now it may turn out that there

01:38:28.980 was no way for me to get your hand right. But I'm supposed to explore that. Because now what I know

01:38:33.960 is that if I knew what your cards were, I would have lost a lot more money than I did. That I actually

01:38:39.520 played the hand very poorly for what the hand was that you actually had. I actually underplayed my hand.

01:38:45.220 And that story, that example you use, I think to me is one of the best examples of why the title of

01:38:50.920 the book makes sense, which is... This isn't a book about poker, actually. It's a book that uses

01:38:56.900 poker as a model system. So for example, let's talk about biology. Why do we do so much biology in

01:39:03.140 C. elegans, a worm, or in this type of a mouse or this yeast? Do we care that much about mice and

01:39:09.800 yeast and worms? No. They are model systems that allow us to do things in cleaner ways. 0.96

01:39:17.660 And I think that's sort of the beauty of poker, which for the listener, they might think they're

01:39:22.000 talking about poker so much. Why? Well, I think you have to understand enough about poker to understand

01:39:26.440 what you just said. Because what you just said is, it's not often in life we will get that much clarity

01:39:32.700 after the fact, but we can still bring that level of critical thought to our decisions. To me, that

01:39:39.600 is the... If you remember nothing else from this podcast, this is the part I'd want someone to

01:39:44.880 remember. I think of poker as the C. elegans of decision-making in life. That's going to be my...

01:39:50.060 I love that. That's great. And we can think about this on the winning side as well, right?

01:39:54.240 Like, I could win a hand from you where I really should have won a lot less. Again, if I had had

01:40:03.800 perfect information, I shouldn't have actually won as much as I did. Because, for example, I could

01:40:09.120 misread your hand, and I could play a hand really strongly, and then I can be the one who hits a

01:40:13.680 lucky card. And actually, if I had perfect information, I would not have ever put that much

01:40:19.640 money at risk. So we want to explore whether the outcome is good or bad. We want to have equanimity

01:40:25.680 toward the direction that we're exploring. Could I have gotten even more out of you? Could I have

01:40:30.200 done even better? Could I have done worse? And the other thing besides that direction, by the way,

01:40:34.160 is we want to think orthogonally. Like, maybe you lost for reasons that had nothing to do with your

01:40:38.080 decision process, or maybe you won for reasons that you totally didn't predict. This happens in

01:40:43.220 financial markets all the time. You invest in a, say, a company because you think that a particular

01:40:49.940 product of theirs is going to do gangbusters or something, and then that product fails, but they

01:40:55.480 happen to acquire another company. By the way, don't take credit for that if that wasn't in your model,

01:41:00.100 but you happen to have won. So we want to be thinking up, down, orthogonal in all these directions.

01:41:05.400 So now here becomes the problem. We know that people don't like to be in this bottom white 0.98

01:41:11.040 chronic, they don't like to be in this place of, okay, something really bad happened. And if they

01:41:16.720 do get in that something really bad happened place, they want to be able to say, but it's not

01:41:22.180 my fault. It's a bad luck situation. It's like, well, my decision process was good, and everybody

01:41:26.900 had consensus, and I handed the ball off to Marshawn Lynch. What could I do? That was what I was

01:41:31.520 supposed to do. So you're driving people into this place where they're very likely to be trying to

01:41:37.140 build consensus, not necessarily of the good kind, right, of the false kind. They may be using data

01:41:42.360 not to find the truth, but to tell a data story that supports them in case they happen to get into

01:41:47.700 the room. So we really like to say the data told us so. They may just be doing what everybody has

01:41:52.640 always done. They may be slow to decide because they need buy-in or the right kind of sign-off too

01:41:57.920 often, or they need a higher level of certainty than they really should in order to make the

01:42:02.300 decision. All sorts of bad things come out of this, and what really ends up coming out of it is

01:42:06.520 that people tend to move slowly. They tend to default toward omissions versus commissions. In

01:42:11.940 other words, not deciding versus deciding, and they tend not to like to innovate too much.

01:42:17.160 Now, my guess is there's a different propensity for this across different fields. So for example,

01:42:22.720 have you ever had a chance to talk to Pete Carroll?

01:42:24.640 I haven't.

01:42:25.480 Just thinking about everything you said, it would be very interesting to have that discussion with him

01:42:29.140 and say, it'd be hard for him to do this, but go back in time to before you made that decision.

01:42:34.220 I'm pretty sure the only thing Pete Carroll was thinking about was winning. I can't imagine there

01:42:39.800 was even an ounce of thought in his mind that said, whatever decision I make, I want to make sure it's

01:42:46.140 defensible at the press conference. I just can't imagine he would have thought that, right?

01:42:49.400 I agree, and I think that's what makes Pete Carroll such a great coach.

01:42:52.440 I was just about to say, but wouldn't you see more Pete Carroll-like thinking in something like

01:42:56.900 sports, or is it less about sports and more about being the head coach versus the defensive

01:43:02.920 coordinator who, in the end, is still making a very important decision, but answers to a coach.

01:43:09.480 Of course, the coach answers to the owner, but I'm not sure I'm articulating the question well.

01:43:13.440 No, I actually think that you are. So first of all, we know across the NFL that the slowness to

01:43:17.780 adopt the analytics has been remarkable. And that's because in this particular case,

01:43:23.560 the people who are judging you are going to be like the GMs or the fans or that kind of thing.

01:43:29.800 And if they're judging you, they're going to be judging you for the unusual decisions.

01:43:34.200 So I think actually, if we go to Belichick, we can kind of see, if we go back to this idea of this

01:43:40.940 feeling of transparency, people are forgiving when there's a feeling of transparency. So if you're

01:43:46.480 Pete Carroll, at that point in his career, people are much more likely, and remember, even with him,

01:43:53.520 they really bashed him, but they're much more likely to sort of give you the benefit of the doubt.

01:43:57.680 So I actually saw a great talk by Toby Moskowitz. He's a really interesting guy. Does a lot of stuff

01:44:03.300 in sports analytics and quant and this kind of thing. And he was talking about Belichick. And he

01:44:09.100 said, when you look at his decision making, when he was with the Browns, it was dismal compared to the

01:44:13.940 analytics. And it's not actually until he wins two Super Bowls that you start to see this decision

01:44:19.960 making that's really starting to align with the analytics. It's really starting to be like,

01:44:25.400 I don't care if nobody's going to understand this decision. I'm Bill Belichick, and that's the

01:44:30.400 decision I'm going to make. But it's after two Super Bowls that he gets there. Why? Because the fans

01:44:35.200 are now tolerant of it. They're willing to tolerate the stuff that they don't understand. Why did you go

01:44:40.140 for it on fourth and five there? That's insane. No, well, because you're Belichick. So what we want

01:44:46.200 to think about is, when we think about these, what you said, the morbidity conference, what is that

01:44:52.620 encouraging in the people that you're trying to lead? You're encouraging the behavior that they

01:44:59.260 naturally bend toward, which is, I want to stay out of this lower right quadrant. So I don't want

01:45:07.040 anything bad to happen to me. But if it does happen to me, I'm going to make sure there's so

01:45:12.300 much CYA there that it's going to be okay, that nobody's going to sit there and say, I made a crazy

01:45:17.500 decision. And you're doing that because you're not, you're not exploring both directions, but

01:45:22.780 particularly because you're not getting in the room when something unexpectedly good happens. And here's

01:45:27.380 the issue. When something unexpectedly good happens, there's so much to learn from that.

01:45:32.320 Is that different, by the way, than when something good happens that shouldn't have happened based on

01:45:38.240 what people didn't know? Now I'm getting sort of Rumsfeld-y in a bit, but okay, I'm making this up.

01:45:43.560 So this could be complete nonsense. So I'm hoping you'll be able to come up with a real example.

01:45:46.820 Remember the whole Y2K thing? Everybody thought, oh my God, this could be a free for all, blah, 1.00

01:45:52.760 blah, blah. Turned out to be the biggest nothing burger in the history of civilization. Like I have

01:45:55.900 friends in college whose entire job after college was getting ready for Y2K. So nothing happened. Fine. 0.85

01:46:02.320 Now there's two ways nothing happened. One is nothing happened because there was really

01:46:06.040 nothing going on and we just didn't understand that. And we did a whole bunch of stuff to be

01:46:10.300 prepared for it. And in the end, it didn't matter. There's another thing, which is, oh no,

01:46:14.000 it was a real problem. And all of that work that was done to make sure airplanes didn't fall out of

01:46:18.460 the sky, it worked. Those are kind of different. And I'm using the problem there is there's work

01:46:23.040 involved versus decision being made. So maybe this isn't a great example. I guess what I'm trying to get at

01:46:26.900 is you can have great outcomes because there was going to be a great outcome regardless of your

01:46:33.300 decision. And then you can have great outcomes where you actually made the wrong decision and

01:46:38.060 you got the right outcome. And then there's, you made the right decision and the right outcome.

01:46:41.940 I'm almost wondering, like, are there other historical examples in wars? I'm trying to think

01:46:45.600 of like D-Day. Was that invasion on that day the right thing to do? I mean, history will forever be

01:46:50.980 changed as a result of it in the right direction. But has anyone ever gone back and said, well,

01:46:55.240 you know what? Based on the weather pattern, like Eisenhower got lucky. That shouldn't have

01:46:59.840 worked that day. And that was the wrong decision. They should have waited a day or something to that

01:47:03.300 effect. Yes. So number one, with Y2K, there's a fourth thing that you can say. So I agree 0.96

01:47:09.000 with all three things that you pointed out. It could have been good decision, good outcome. It could have

01:47:14.480 been, it would have happened no matter what we did. And so we were irrelevant. It could be that we had a

01:47:20.520 good outcome and it was all sort of a waste of time. So it was a bad decision to do anything.

01:47:24.040 It could be it doesn't really matter. Because when you sort of look at what the range of

01:47:28.200 possibilities were given the unknown information, the downside was so bad that you were actually

01:47:33.980 supposed to protect against that regardless. And were you going to waste a whole bunch of human

01:47:38.400 productivity doing it? Well, it's not really a waste because it just acts as a hedge against

01:47:42.240 just in case it's the worst case scenario. And particularly with something like that, we would

01:47:46.180 want to protect ourselves against that. But unless you dig into the win, you don't find that stuff

01:47:52.060 out. And I think that that's where the problem is. So with the Eisenhower example, what you said,

01:47:58.020 it's such a great model for how do we think about our own outcomes in our own lives. So this is what

01:48:03.480 tends to happen to us. And this is why I was saying, oh, it's actually not painful at all when you have a

01:48:09.140 bad outcome to go in and look. It's what we naturally do. Something horrible happens to me. I'm looking in

01:48:15.680 there. I'm trying to find all the luck in there. I'm exploring what were my other options? Was there

01:48:20.460 something else that could have worked out better? How often was that decision going to work out

01:48:25.080 poorly? And I'm so excited. I'm so excited to go in and discover that I didn't do anything wrong.

01:48:30.940 I'm looking for that when it's a bad outcome, when it's a good outcome. When we land on the beaches

01:48:37.120 of Normandy, and it's a pivot point in the war, and we win, people aren't looking as hard to try to

01:48:45.040 figure out to really explore the counterfactuals. Well, what if I had done this? What if I had done this?

01:48:50.460 What if it had turned out this way? So one of the first steps to being a really good decision

01:48:56.800 maker, because we have this asymmetry in how willing we are to explore these outcomes.

01:49:01.100 And it's what you just said. It's we're all living our lives having a morbidity conference.

01:49:06.280 Now, the morbidity conference that we're having in our head is specifically to go find the luck in

01:49:10.740 the whole thing. But that's the only conference we're having. We're never having the, my life is

01:49:16.240 great conference. Let me go figure out if that's because of my own decision making, or because of

01:49:20.880 luck, or if there was a better way, or I could have actually opened up a whole other set of

01:49:24.500 outcomes. Like nobody's, nobody's doing that.

01:49:27.120 That is such an interesting point. You are so correct in that. And as you're telling that story,

01:49:31.800 I'm thinking of a sort of silly example in my life. But it's one that anyone who knows me knows

01:49:37.020 I take very seriously, which is how much I love archery. And I shoot a lot. So when I am not in New York,

01:49:43.100 obviously, when I'm back in San Diego, I'm shooting constantly. And I'm always trying to push the

01:49:47.660 limit of my capabilities, which means you're losing arrows. You're just invariably, if you're 60,

01:49:53.720 70 yards away, trying to hit something the size of a softball, you're going to miss every once in a

01:49:58.340 while. I'm generally losing an arrow to the tune of one a day. Now, these arrows are very expensive.

01:50:03.280 So this is becoming quite a problem. Every one I've missed, I can tell you a hundred reasons why,

01:50:09.740 and not bad luck. I can tell you the mistake I've made. I don't think I've once ever given thought

01:50:17.440 to the perfect shots. I don't think I've ever spent time thinking about that ever. I think

01:50:24.060 90% of my energy goes into thinking about the 10% of the shots I didn't like. And I thought that's

01:50:30.560 the right way to do it. Isn't that making me a better shooter? What I'm hearing you say is that's

01:50:34.800 part of making you a better shooter.

01:50:36.080 It's also going to encourage, I mean, if you think about sort of societally, if it's the bad

01:50:40.460 outcomes that are triggering the dives, first of all, you're losing half of the opportunities to

01:50:44.940 learn.

01:50:45.240 If not more than half, because you might even be making more positive outcomes than less.

01:50:49.620 Right, right. Well, let's assume that the chances of having an unexpectedly good outcome are symmetric.

01:50:55.360 And let's make it that the thing that triggers us doing a dive is that it's unexpected. And you can

01:51:02.640 decide if it's like a one standard deviation away or the standard deviation and a half or whatever it

01:51:07.660 is, we can decide and that would obviously depend on the uncertainty and how much ball there is and

01:51:12.180 so on and so forth. But we'd figure that out. And let's assume it's symmetric. So by only digging into

01:51:16.780 the downside, we're doing two things. One is we're losing half of our opportunities to learn.

01:51:20.380 And the second thing is that we're reinforcing the risk aversion. We're saying, hey, what we really

01:51:26.620 care about is downside outcomes. So you better avoid those. And three is we're completely quashing

01:51:32.120 anybody ever doing anything unexpected, where, because if you do something unexpected, the chances

01:51:37.320 that somebody is going to allow for luck or that you yourself are going to allow for luck is the

01:51:41.040 explanation is just going to kind of go poof. And we don't like that. So there's all sorts of bad

01:51:45.360 stuff that comes out of it. So when we get an outcome, we want to basically do what you were

01:51:50.480 just doing with the beaches of Normandy, which is to say, here's the outcome. Before I make any

01:51:56.980 conclusions about it, let me think about what are the other possible things that could have occurred.

01:52:03.120 So I have to think about what are the things that didn't happen but could have those counterfactuals.

01:52:08.360 And then I need to think about what was my preference for those things? How much did I like each of

01:52:13.860 those things? And how often were those things going to happen? So it's just basically thinking

01:52:18.060 about what was the payoff for those things. Obviously, you can compare those to cost. But

01:52:21.380 let's just simplify it. What were the possibilities? What were my preferences for those possibilities?

01:52:27.920 And what was the probability of each of those possibilities occurring? And then, so that's the

01:52:33.720 first step with the beaches of Normandy. It ended up being a particular way. But given that they decided

01:52:39.260 to do what they did on that particular day, what are the other ways that it could have turned out

01:52:43.340 with what probability would have that have turned out? And how much would we have liked or those

01:52:47.840 things depending on what our goal was, which in this case was to win the war. And then now we can

01:52:52.040 take that and we can iterate it and say, what if they had waited two days? What if they had done it

01:52:58.400 two days before? What if they had done it in a different way? They had gone from land instead of sea.

01:53:04.620 Now we can start thinking about if they had chosen other options, then how would that have compared

01:53:10.840 in terms of what the expected value was for the option that they actually chose? And you can do

01:53:16.460 this for anything. So you can do this for the archery. I assume that a bullseye is pretty

01:53:19.720 unexpected. At certain ranges, it's more surprising than it is not. Right. What you can say is you could

01:53:24.520 actually go in and start forecasting it. I'm going in at a particular range, given what I think that

01:53:28.840 my skill level is, what percentage of the shots do I think I'm going to get a bullseye? So forecast it.

01:53:34.260 Say this is the percentage of times that I think I'm going to get a bullseye. Now, if it's a particular

01:53:38.240 day and you're well below that, you would say, hmm, I wonder what happened today. What was the

01:53:43.740 situation today where I was well below what my expectation was for the number of bullseyes?

01:53:48.520 But now, if you're way above, you do the exact same thing. And this is what we don't do. I can

01:53:55.160 guarantee you, because I know this is the way that people think. If you think, oh, I don't know how many

01:53:59.480 times you shoot at the target. I've done so much of the work you're describing without the important

01:54:03.900 piece. But at 50 yards, for example, I know, just based on what I do, that I should be able to put

01:54:10.580 8 out of 10 inside of a certain diameter. So I spend a lot of time thinking about when I'm hitting

01:54:18.220 6 out of 10 inside of that range. What's going on? But there's days I'm hitting 10 out of 10.

01:54:22.700 This is what you do. You walk off and you dance. You're like, look at me.

01:54:25.840 Yeah, I don't think about why.

01:54:26.880 You're like, whoa, check me out. I'm great. And the thing is that, of course, you may have

01:54:33.500 stepped up in skill, but wouldn't you like to know exactly what the adjustment, what was it that

01:54:37.380 actually clicked for you that day? What is it that changed?

01:54:40.000 And I know it's not that I stepped up in skill because it can vanish the next day. So it's clearly

01:54:44.300 something that is transient.

01:54:46.160 Well, one of the reasons why it may be transient...

01:54:48.500 Is because I haven't identified it and marked it and incorporated it.

01:54:51.740 Right, because it could be totally transient. It could be that you get to a certain skill

01:54:55.480 level and this is going to be your range. It's going to be 6 to 10.

01:54:58.160 Oh, well, by transient, I just mean what I'm really saying is I'm sure that I did something

01:55:02.580 better on that day, but it's transient because I didn't identify it and make it a part of my

01:55:07.480 permanent... It could be that if we could look at me under the 6 out of 10 day versus the 10 out of 10

01:55:12.400 day. I mean, I'm positive I'm doing something different. There is no way equipment, wind, or any

01:55:18.660 other factor other than me explains that. But by only comparing 8 to 6, not 6 to 10,

01:55:25.340 I'm missing a greater disparity, which in theory should call out more obvious deltas. It's not

01:55:32.040 just that you're missing half the time, it's you're using half the discrimination. It's like it's easier

01:55:36.680 to see that there are two lines if they're a foot apart versus a millimeter apart.

01:55:40.980 So if I had to say, like if someone asked me what's the single biggest factor between a player who

01:55:48.620 doesn't make a ton of progress or makes slower progress compared to a player who really,

01:55:54.920 really makes a lot of progress and becomes elite in poker, I would say that's it. That when you

01:56:00.340 listen to someone like Eric Seidel talk about poker, you would hardly know whether they won or

01:56:06.800 lost the hand. All they're doing is exploring what are the counterfactuals? Like what are the other

01:56:12.440 lines that I could have chosen? Did I have the person's hand right? What if I had bet a little bit

01:56:17.300 more? What if I had bet a little bit less? What if I hadn't played the hand at all? What would have

01:56:22.000 happened if I played this hand that I didn't play? And you would hardly know. Like if you listen to

01:56:26.020 them, you would have no idea. In fact, probably you would assume they had lost. Right, because it's

01:56:29.800 so foreign to hear somebody. Because they're exploring so much. You'd be like, whoa, you must have

01:56:34.440 really lost that tournament. And they'd be like, no, I won the tournament. And you move on. And what you

01:56:40.020 hear from players who aren't making that kind of progress is it goes two ways. When they talk about

01:56:44.740 the hands they won, it's just sort of like I was great. And they don't really spend a lot of time

01:56:48.720 on it. When they talk about the hands they lost, it's a lot of exploration of the luck elements.

01:56:53.780 And the thing about poker, because the luck element in a given instance, like in the short run is so

01:56:58.940 prominent, that it's so easy to focus on the bad card that came. So you'll have conversations with

01:57:08.220 people where they'll talk about like, oh, I had this hand. And this person, it's called sucking out. 0.92

01:57:14.620 They sucked out on me, meaning that they happened to hit a good card. And to somebody who's thinking, 0.98

01:57:20.420 who spent a lot of time around someone like Eric Seidel, you're looking at it and saying,

01:57:24.920 well, you should have won that hand way beforehand. Or maybe you shouldn't have played that hand at all.

01:57:29.120 So for example, like if I have a particular hand where you end up hitting a really good card,

01:57:33.800 it may be that I should have made you fold before then. Now sometimes it's you were going to be

01:57:38.420 sticky, you were going to stay around, I had mathematically way the best of it. And whatever,

01:57:43.440 bad luck happened. But sometimes it's I should have made you fold a lot earlier,

01:57:48.200 I should have made you pay more for the privilege of hitting that card, that actually the way that

01:57:54.220 I bet the hand meant that you were making money to my bet when I could have made you lose money to

01:57:58.880 my bet, regardless of whether you were going to beat me on that particular hand in the long run,

01:58:02.480 I'd like to make you make losing decisions. Maybe I shouldn't have played the hand at all.

01:58:07.280 There's all sorts of other things that you can be exploring in those situations,

01:58:10.880 and they're not getting explored. Instead, what's getting focused on is I can't believe

01:58:14.460 that person sucked out on me.

01:58:16.080 So I have this framework for learning things that require skill. And I see a parallel now that

01:58:21.180 I've never seen before. So my framework, and I use swimming as the analogy for this,

01:58:25.320 because I learned to swim as an adult. So this was really the first time I explored this framework.

01:58:29.580 But I do think it applies to many things, languages, you pick it, frankly.

01:58:33.500 So the framework is we start out as unconsciously incompetent. So you take a person like me at 31 who

01:58:42.140 doesn't know how to swim, you put them in the water. There's no ambiguity about how bad I am.

01:58:46.660 But I don't know why. All I can tell you is I can't swim. I can't actually tell you why I can't swim.

01:58:51.840 So the first step of learning how to swim is becoming consciously incompetent. You actually have

01:58:57.140 to understand why you're really bad at this. Oh, once you start to understand Archimedes principle,

01:59:04.040 that's like a very important part of understanding how to swim. The more of you that's underwater,

01:59:08.560 the more buoyancy there is to hold you up. Once you start to understand how Bernoulli works,

01:59:14.260 oh, all of a sudden, there's this difference about the velocity of a fluid over a body, etc.

01:59:19.360 And then by the way, you're no better as a swimmer in those two boxes, right? The difference

01:59:23.120 between... Right. I just want to know the percentage of people who are learning swimming

01:59:27.140 who cite Archimedes and Bernoulli. I just love that that's the way that you're... I don't know if I

01:59:33.340 still have them, but I would love it if I did. I used to journal so relentlessly about swimming.

01:59:38.200 I had reams of journals that I would write in every single day of my free body diagrams of my body in

01:59:46.960 different swimming positions and trying to figure out ways to make this less bad. When I got into this,

01:59:53.120 I became so obsessed that I swam twice a day, every day. This model makes so much sense to me.

01:59:58.520 Well, you don't actually start swimming until you reach the third box, which is now you are

02:00:03.400 consciously competent. Now, the problem is you can't do that for long. So you tend to vacillate a lot

02:00:10.440 between the second and third box, which is consciously competent. And then you go back to being

02:00:15.640 consciously incompetent. But as long as you can maintain focus, you will vacillate between those.

02:00:21.340 And usually it's your physical fatigue that will push you from the third box back to the second

02:00:26.920 box. Now, when you become a really good swimmer, like Michael Phelps, you transcend to this fourth

02:00:32.540 box. You are now unconsciously competent. Everything becomes sort of autonomic at that level. Now,

02:00:39.500 thinking about this through decisions, as you were telling the story about Eric, I'm thinking about

02:00:43.640 the first level. So let's just call it four levels. Now, the first level is you only examine losses

02:00:49.760 through the lens of luck. So that's our default. I would argue that takes no effort, just as being

02:00:55.800 unconsciously incompetent is our default. Layer two or level two thinking is I only examine my losses

02:01:04.480 through the lens of luck and skill. That's great.

02:01:08.800 So you're taking some responsibility, but it's only triggered by the loss.

02:01:11.620 Only triggered by the loss. Then you have a level three, which is I'll do that,

02:01:15.720 but I will also now examine my wins through skill. And then the fourth level, which I'm guessing is

02:01:22.720 exactly where someone like Eric is, is no, I'm going to look at wins and losses through luck

02:01:28.780 and skill.

02:01:29.840 Yeah. So what I would say is I think that the thing about examining wins, I do think that examining

02:01:38.580 wins through the lens of skill is pretty natural. We don't spend a lot of time on it. It's kind of an

02:01:44.660 assumption of skill. I would put wins down in level one. Let's put it this way. I would focus

02:01:49.680 more on, I would actually think about it, not just in terms of, are you examining the quadrants that

02:01:58.260 are discordant in a symmetric way? So what I would say is that on the level one thinking is I lean on

02:02:07.440 the discordance on the loss side and I lean on the concordance on the win side. And I just kind

02:02:13.660 of do that naturally. And I don't spend a lot of time thinking about either of them.

02:02:17.520 That makes sense.

02:02:18.480 I don't spend a lot of time thinking about either of them. Then I think that level two is you're still

02:02:24.620 leaning on concordance on the win side. You're not examining those very much, but you do start to

02:02:28.940 take the losses.

02:02:29.040 At least you examine the discordant losses.

02:02:31.020 Right. You examine the concordant losses now. So you examine the losses that are actually due to

02:02:36.040 your own undoing. Then I'll get to the fourth level. Then I think that level three is I'm looking

02:02:45.980 in all four quadrants. So I'm looking at wins that are due to skill, that are due to my good

02:02:51.340 decision making. I'm looking at wins that are due to my bad decision making. I'm looking at losses

02:02:56.540 that are due to my bad decision making. I'm looking at losses that are due to my good decision making.

02:03:01.280 And I'm trying to, I'm really trying to sort of look at those all equally.

02:03:04.760 Where I think the fourth level comes in is when it's concordant on the win side that

02:03:13.420 you don't accept it.

02:03:15.120 In other words, that you say, I made good decisions and I won, but it does not mean

02:03:20.960 there wasn't a better way.

02:03:22.660 So let me actually start to really clean up in there and say, it's not about I made a

02:03:30.040 decision.

02:03:30.380 It was a winning decision.

02:03:31.740 I was positive expected value for it.

02:03:34.380 And so I'm done.

02:03:36.180 It's now I so care that I got onto what we would call the primary line of play that I

02:03:43.000 really came up with the best possible solution that I could given what I knew that I won't

02:03:48.320 be satisfied with just finding out that there was concordance.

02:03:51.040 Like I want to go beyond that on the win side.

02:03:53.040 I think what we're going to do in the show notes is literally take everything you just

02:03:56.280 said and turn it into a printable one pager that every one of us should carry with us.

02:04:01.680 Because that's that is a very elegant way to think about that.

02:04:06.220 I promise you I've never done what you just said.

02:04:08.900 Like I've never once had a level four thought in my life.

02:04:12.040 Well, first of all, let me just say that I only thought about it because you gave me

02:04:16.080 the levels.

02:04:16.580 But it's that last bit.

02:04:18.380 So right now I'm writing a workbook, very close to finished with it, where in chapter

02:04:23.620 three, I really dig into that piece, which is when you have a good outcome and it turns

02:04:30.520 out it's from good decisions, what are you doing with that?

02:04:33.400 Are you done?

02:04:34.540 Or are you trying to figure out if there was even better one beyond that?

02:04:38.320 And here's the reason why we don't like to do that.

02:04:40.460 Because when we think about what sort of creates the fabric of our identity, our beliefs are

02:04:47.760 the threads that our identity is woven out of.

02:04:50.380 By doing a level four thought, you risk they're undermining.

02:04:52.860 Tearing a hole, tearing a hole in it.

02:04:54.820 Because one of the things that we want to do is feel like that fabric is strong, like

02:04:59.420 our beliefs are good and we're competent.

02:05:02.740 I mean, this idea of competence and being able to bank credit for the way that things have

02:05:09.060 turned out in our life is so core to what forms our identity that when you don't stop

02:05:15.740 at, hey, my decisions were good and I had a good outcome, when you actually dig around

02:05:21.220 in there, you're risking now turning something that was a win into a loss.

02:05:27.140 So this is where I love your idea of the levels.

02:05:30.140 So if we think about that level one, like I won, it must be good.

02:05:33.040 Why wouldn't you go further than that and say, well, maybe I won because of bad luck.

02:05:36.600 You're turning a win into a loss.

02:05:38.280 And in this particular case, we're taking win as the outcome.

02:05:41.120 I had a good outcome and you sort of want to just stop there because you don't want

02:05:44.900 to dig down and find out you might have made a bad decision because now you turned a win

02:05:48.960 into a loss.

02:05:49.880 So that's where when you get to level two, you're willing to do that.

02:05:53.380 You're willing to say, I want a long run win.

02:05:56.340 And what that means is I might have to turn this outcome that was a win into a loss.

02:05:59.980 When you get to level four, what you say is, okay, I had a good outcome.

02:06:04.780 I looked and I saw that my decision making was pretty good.

02:06:07.520 But now I'm willing to turn that win into a loss in order to get long run better.

02:06:12.160 You were basically completely exposed at that point.

02:06:15.060 I can imagine how your book became immediate fodder for every hedge fund manager and so

02:06:20.820 forth.

02:06:21.020 Because again, I think the advantage of that type of investing bears some resemblance to

02:06:26.680 poker, which is you can't really hide your decisions.

02:06:30.040 Your outcomes are very clear and which is not to say there's not a huge element of luck just

02:06:35.300 as there is with poker, but it's basically an industry where your decisions are exposed.

02:06:40.520 Every decision you make, in the end, you're going to be able to trace it to an outcome.

02:06:45.300 And frankly, unless you're not in the business of making money, you have to be wed to good

02:06:50.140 outcomes.

02:06:50.900 So the poker player wants to do well.

02:06:52.920 The hedge fund manager wants to do well.

02:06:54.680 There's nowhere to hide, I guess, is what I'm trying to get at.

02:06:56.940 I agree in theory that there's nowhere to hide.

02:07:00.080 The interesting thing that I think is how long people can hide from it.

02:07:06.300 That's what I find so fascinating is you look at poker and you're like, you're getting answers

02:07:11.720 right away, right?

02:07:12.640 Like you win or lose.

02:07:14.060 So here's the problem.

02:07:15.300 It's like, okay, at the end of the year, somebody lost, right?

02:07:19.040 But why?

02:07:20.220 How far are you willing to go to say, I just got bad luck?

02:07:24.540 That's the question.

02:07:26.020 And I'm just really fascinated that people can go really far.

02:07:31.040 They can really convince themselves that they just got bad luck.

02:07:34.960 And I think that that's true.

02:07:37.240 Obviously, if you're running a hedge fund, you have investors that you're accountable

02:07:40.740 to.

02:07:41.540 But we all know of hedge fund managers where they had a couple of good years and then

02:07:46.400 people are sticking with them.

02:07:47.620 It's like you have a couple of good years to start.

02:07:49.440 There's this path dependence to this.

02:07:51.020 And it's true of poker.

02:07:51.620 If a poker player comes out of the gate in the first six months and loses everything

02:07:55.080 in sight, they're very unlikely if they continue to lose.

02:07:58.620 They're going to say, oh, I must be really bad at this.

02:08:00.820 But for every poker player who comes out of the gate with a terrible first six months,

02:08:04.500 there's a poker player.

02:08:05.700 I actually remember way back when, a long time ago, there was a new player who came

02:08:11.340 on the scene and they won a World Series of Poker bracelet.

02:08:15.880 I mean, they were really young.

02:08:16.880 I mean, early 20s.

02:08:18.080 And nobody had ever heard of them before.

02:08:19.840 And they won.

02:08:21.080 And I thought, oh, you know, that's nice.

02:08:22.280 Because that happens, right?

02:08:23.180 Obviously.

02:08:23.920 Then they won a second one.

02:08:25.360 And I was like, oh, that might be just the ruin of them.

02:08:28.720 Now, why would I say that?

02:08:29.700 Like, they had won a second World Series of Poker bracelet.

02:08:31.900 Because I understood this path dependence, right?

02:08:34.580 That if you come out of the gate and you do really well, your ability to fool yourself

02:08:40.200 afterwards for a very long time.

02:08:42.040 It becomes infinite.

02:08:43.080 It becomes almost infinite.

02:08:44.040 And you see that in the hedge fund world all the time.

02:08:46.360 And obviously, if an investor comes along and they don't show any results, by the way,

02:08:50.500 sometimes unfairly, it becomes very hard for them to recover from that.

02:08:54.400 Sometimes unfairly, because maybe it just went against them for that year or whatever.

02:08:58.720 But when it's the reverse, when they come out of the gate strong, you can fool yourself

02:09:03.240 for a really, really, really, really long time.

02:09:05.460 And that's the thing that I find so fascinating.

02:09:07.660 That's what really kind of lights me up.

02:09:10.800 And trying to figure out the capacity for self-deception is so a bottomless well.

02:09:17.460 Is there an industry that you look at where you just see, on average, it is more amenable

02:09:23.440 to higher level thinking the way we just described the four levels in poker, like this framework?

02:09:30.020 Is there a place where, either through some externality of the way the industry is set up

02:09:35.460 and the rate of time being short between decision and outcome and some sort of transparency,

02:09:41.380 that we have a space where there's a greater incentive to rise to that level?

02:09:45.660 First of all, let me just say this, that obviously, we're only talking about places

02:09:49.880 where, in the shorter run, luck is going to have a strong influence.

02:09:53.620 So when you look at sports that are really highly under the influence of skill, people

02:10:00.520 are getting selected out.

02:10:01.920 Nobody's fooling themselves.

02:10:03.860 Well, actually, it was just bad luck that I didn't get to be a professional NFL linebacker.

02:10:09.780 I mean, it might be if you got an injury, but I'm 5'5 and I weigh 130.

02:10:14.540 Such bad luck that I wasn't.

02:10:16.340 People are just like, no, I just wasn't good enough.

02:10:17.980 So we know that in cases where it's very transparent that if you're not doing well,

02:10:23.980 we can trace it back to skill.

02:10:25.540 So the more chess-like we are, the less the capacity for us to fool ourselves.

02:10:30.580 If I lose to you in chess, it's very hard for me to say that I got bad luck because,

02:10:34.620 I don't know, where's the luck?

02:10:35.700 I don't know, go find it for me.

02:10:37.060 There's a little bit of luck, but not much.

02:10:39.220 So the more on the chess range of things we are, the less the capacity to do that.

02:10:44.100 So first of all, we've got to be in an environment where there's a strong short-run influence

02:10:47.840 of luck.

02:10:49.040 Now, the tighter the feedback loop, the more feedback you're getting, the more that somebody

02:10:55.600 needs to rise to the occasion.

02:10:57.240 So if you were to look at what's the capacity, which individuals are going to thrive more

02:11:01.880 in those systems, definitely the ones who are much more willing to dig in and dig around.

02:11:07.680 So people who are in higher frequency trading, for example, where those feedback loops or the

02:11:12.720 time horizons are shorter, I think are going to be more likely to be thinking in this way

02:11:17.820 and if they're investing their own money.

02:11:21.040 Ah, that's a very interesting point.

02:11:22.600 It's got to, Nassim, right?

02:11:23.900 You got to have skin in the game.

02:11:25.020 Are there poker players, by the way, that play with other people's money?

02:11:27.360 And by that, I mean not money that they won from somebody else, but I see.

02:11:31.340 Okay.

02:11:31.600 Yeah.

02:11:31.840 So there's backing situations.

02:11:33.160 There's backing.

02:11:33.340 Okay.

02:11:33.440 Yeah.

02:11:33.640 There's the same thing.

02:11:34.540 The less you're trading on sort of reputation and the more that you're trading on your own,

02:11:38.260 your own dollars.

02:11:39.300 Also, I think that you're more likely to end up in a spot where you're thinking that way.

02:11:42.900 So basically you have tons of skin in the game.

02:11:45.660 There's a high element of luck and skill and the feedback loop is short.

02:11:50.720 Those are three situations that will tend to push you towards either abject failure or getting

02:11:57.900 better, moving up the ladder of this thinking.

02:12:00.180 It's a little chicken and egg.

02:12:02.760 So I don't know.

02:12:03.960 Is it the people who succeed are more naturally, kind of more naturally tend to that kind of

02:12:09.320 thinking and obviously it gets reinforced or.

02:12:13.200 What would Eric do if he wasn't the best poker player in the world?

02:12:16.660 Well, I know what he would do because he was an options trader.

02:12:19.720 Okay.

02:12:20.260 So again, options trading, you could argue that it's sort of similar, right?

02:12:23.820 It's incredibly similar.

02:12:24.980 It's almost the same.

02:12:26.040 But would he go and be a good CEO of a blue chip company?

02:12:30.380 Like if he all of a sudden tomorrow, assume he had the domain knowledge.

02:12:33.880 Okay.

02:12:34.120 So he could go in and be the CEO of General Electric or Home Depot or Exxon Mobil or something

02:12:40.100 where you could put a chip in his head that gives him the domain expertise to understand

02:12:44.340 exploration and production or whatever.

02:12:47.360 But could you move that industry or that company within its industry relative to its peers by bringing

02:12:54.420 that layer of thinking?

02:12:56.160 Yes.

02:12:56.500 So that's what, that's, that's kind of what I want to say.

02:12:58.760 Is there a little bit of a chicken and an egg issue?

02:13:01.000 Because I have met people across industries where there are people who are thinking this

02:13:05.560 way, where there are people who are really hungry for it.

02:13:07.840 Now, in terms of this idea of like, what are you doing when you find out it was pretty

02:13:11.820 good decisions and good outcomes, or are you only doing postmortems and not thinking

02:13:16.740 about it in terms of expectancy?

02:13:18.420 That's a problem that you see across all industries.

02:13:20.580 But also across industries, you see people who are really hungry to try to incorporate

02:13:25.540 that into their solution space.

02:13:28.640 How am I dealing with this?

02:13:29.660 How am I running my teams?

02:13:31.420 So if you just take the investing world, I see people who think that way all the way

02:13:36.480 from people who are trading options to people who are doing seed round venture capital, much

02:13:43.340 different than late stage venture capital where you might, the time horizon now is faster.

02:13:48.120 I mean, not the same speed as options, but if you're like in the seed round, we're talking

02:13:52.540 horizons, it could be a decade.

02:13:54.340 There are fund managers who are in long short funds where they might put on seven positions

02:13:59.380 in a year.

02:14:00.180 You're hardly getting any data.

02:14:01.860 And they're, they're really looking to think this way because what's interesting is that

02:14:07.040 while trading options might foster this kind of thinking better because you're getting

02:14:13.500 feedback more quickly, while you might end up getting sort of selected out by the people

02:14:18.720 who you're accountable to quicker, while maybe that particular type of mind would tend

02:14:23.700 to get drawn toward, say, options trading better.

02:14:27.240 The issue is that in absence of the world telling you quickly, this type of thinking actually

02:14:33.260 becomes much more important.

02:14:35.180 Yeah, it becomes more important and more difficult.

02:14:37.240 Right.

02:14:37.480 Because when you spread the feedback loops out now, the way that I always think about

02:14:42.660 it is uncertainty gives you the leeway to allow bias to come in.

02:14:49.100 And that's the difference between the chess and poker.

02:14:51.140 In chess, because you've taken a lot of uncertainty away, you don't have the leeway to fool yourself.

02:14:55.780 That is an absolute gold point.

02:14:57.300 But in poker, you do.

02:14:58.200 So we can sort of think about if you look at the difference between, say, trading options

02:15:01.520 and something that has a much longer time horizon, the longer the time horizon, the more capacity

02:15:06.620 you have, the more leeway the world is giving you to fool yourself.

02:15:11.740 So then this kind of thinking, really thinking in this systematic way, really modeling the

02:15:17.040 world in this way becomes much, much, much more important.

02:15:20.180 It becomes a bigger imperative for you to be poking, for you to be thinking counterfactually,

02:15:25.060 for you to have people challenging your ideas, for you to be trying to think about, am I doing

02:15:31.340 this because it's status quo?

02:15:34.200 And if it doesn't go well, I won't get blamed.

02:15:37.700 Am I innovating enough?

02:15:38.900 Am I allowing people to challenge my ideas?

02:15:40.900 Am I challenging my ideas myself enough?

02:15:43.760 Am I looking at that discordant box that has to do with good outcomes?

02:15:50.180 Or am I only willing to look at the discord when it's a bad outcome?

02:15:53.720 When the bad outcome is concordant, am I like super excited about that?

02:15:57.840 Or am I imagining, well, maybe there was a better way?

02:16:00.880 How am I digging into all of those spots?

02:16:03.220 And then if you're in a leadership role, how am I propagating or am I not propagating these

02:16:08.760 problems across the team?

02:16:10.600 Am I telling the team through my actions that they should be terrified of bad results?

02:16:15.480 As I'm thinking about the way that the world is giving me the leeway to be able to sort of

02:16:19.500 have my beliefs lead me around on a leash, am I infecting my team with that same problem?

02:16:26.440 Because now I can be creating this whole other problem.

02:16:29.180 And it's such a long time horizon that maybe the world's going to tell me that this was

02:16:33.280 actually problematic.

02:16:34.860 You should be digging into that as if you're getting the answer tomorrow, as if you're playing

02:16:39.060 chess and you're going to find out right away.

02:16:40.980 And you should be doing everything you can to try to make sure that you don't find out

02:16:46.640 you hung your rook.

02:16:48.040 This to me is why swimming is harder than riding a bike.

02:16:50.460 You can teach anybody to ride a bike relatively quickly for a couple of reasons.

02:16:53.640 One, the feedback loop is very quick.

02:16:56.700 Like anytime you're out of balance, I mean, we just naturally internally, our inner ears

02:17:01.060 allow us to sense that disequilibrium instantaneously.

02:17:05.120 So you get the fastest feedback loop imaginable and the consequences are quite high.

02:17:10.180 It hurts significantly when you're not in balance.

02:17:12.720 Contrary to popular belief, swimming is about balance.

02:17:15.040 It's the exact same thing.

02:17:16.220 You just happen to be in a medium that is so much more dense than air that the feedback

02:17:20.920 loop is much slower and the consequences are much more gentle.

02:17:25.100 It doesn't hurt when you're out of balance.

02:17:27.640 In fact, if you fell every time on a bicycle that you were out of balance while learning

02:17:31.840 to swim, you'd never ride a bike again.

02:17:33.680 It would, you'd be so black and blue.

02:17:35.520 So in that sense, swimming is gentler, meaning you have more excuses.

02:17:40.420 It's longer time horizon, but it's that much more important that you do the examination.

02:17:45.760 So if anybody can mindlessly ride a bike very soon after being shown how to do it, you can't

02:17:51.720 mindlessly swim for years after learning how to swim.

02:17:55.320 It's that hard.

02:17:56.700 Swimming then becomes the example of where it's more important that you bring that level

02:18:02.240 of thinking if you want to make any progress.

02:18:04.960 I agree.

02:18:05.680 And let me add something to this, which correct me if I'm wrong, but I think this is in the

02:18:10.280 same space as you're thinking.

02:18:11.440 Sort of thinking about the soft landing issue, the wider the skill gap between you and the

02:18:21.100 people that you're competing against, there's so much more cushion for a soft landing.

02:18:26.680 So this is something that I sort of think about is by not exploring that, sure, I made winning

02:18:34.140 decisions and I won.

02:18:35.720 By not willing to go beyond that.

02:18:37.820 Imagine if you're in a situation where, let's say we're in some sort of zero-sum game, so

02:18:42.840 we're trading against each other.

02:18:44.900 And if I were making perfect decisions against you, every time that we put a dollar on the

02:18:52.280 table, I would get a quarter.

02:18:54.600 But because of the decisions that I'm making, every time we put a dollar on the table, I'm

02:18:58.740 getting a nickel.

02:18:59.860 So if we think about those quadrants, right, when I look at I have a good outcome, I won a

02:19:03.520 nickel, and my decision-making was good because I was positive expectancy, right?

02:19:08.360 I was winning a nickel.

02:19:09.500 I was winning 5%.

02:19:10.640 But by not exploring, I don't see the 20 cents that's sitting there that I could get every

02:19:17.340 single time.

02:19:18.500 And that's the problem with a soft landing.

02:19:20.900 That's the problem with having that cushion, is it doesn't let you know that there's something

02:19:26.700 better that you could be doing.

02:19:27.960 I know we got to wrap because we both have to get moving, but there's one topic that

02:19:31.800 you very loosely, I think we were going to go down the path of it by name, but then we

02:19:36.800 didn't.

02:19:37.360 But I want to bring it up because it's such an important concept in my world.

02:19:41.100 And it's something I have done for such a long time, but never had a great name for it.

02:19:46.780 And when I read your book, I was like, oh, awesome.

02:19:49.740 I'm going to be plagiarizing this all day long.

02:19:52.600 And it's the idea of backcasting.

02:19:54.400 Oh, yeah.

02:19:54.940 A moment ago, you used the term forecasting.

02:19:57.260 Right.

02:19:57.540 We're all about forecasting.

02:19:59.000 Well, I want to be doing this in Q4, and therefore I need to do this today and this tomorrow and

02:20:04.580 this tomorrow.

02:20:05.200 And I'll tell you how I use it, and then I'll...

02:20:08.040 And let me just say for the record, backcasting is not a word that I coin.

02:20:11.900 Oh, all right.

02:20:12.760 I don't want to take credit for that.

02:20:13.920 I've got to reference you who references someone else.

02:20:15.140 You can say you read about it in my book.

02:20:16.860 I didn't invent that word.

02:20:18.080 All right.

02:20:18.340 Thank you for clarifying that.

02:20:19.780 So prior to reading your book, I would just refer to it as reverse engineering, but that never really

02:20:24.400 had any resonance with people.

02:20:25.620 But now when I say backcasting, it forces me to explain the contrast between backcasting

02:20:30.260 and forecasting, whereas reverse engineering doesn't have a striking of a contrast.

02:20:35.080 So the place that I use it is in understanding the exoskeleton of the body.

02:20:40.080 So the actual physical structural demise of ourselves.

02:20:44.640 So we're all sitting here thinking about exercise and everybody knows they should exercise, but

02:20:48.420 nobody really can give you a great reason for why.

02:20:50.460 We've been sitting here talking about the outliers, professional athletes, but outside of people

02:20:54.760 who get paid to do something with their body, what are the rest of us doing?

02:20:58.940 It's amazing how often you push people on this.

02:21:00.980 They can't give you a great answer.

02:21:02.280 And because I ask everybody under the sun, this question, I'll tell you all answers basically

02:21:06.200 fall into these three buckets.

02:21:07.520 So if you're asking someone who's not getting paid to do a sport, why do you exercise?

02:21:13.020 90% of the answers are, I like the way it makes me look.

02:21:16.420 I like the way it makes me feel.

02:21:19.280 And I like the freedom and flexibility it gives me with what I eat.

02:21:23.440 Some combination of those three.

02:21:24.440 There are other answers that show up a lot less frequently, such as I love the social interaction

02:21:28.160 that comes with it and things like that.

02:21:29.320 Now, I would argue all of those are fine reasons to exercise, but I don't think they're

02:21:33.360 good enough.

02:21:33.960 The good enough one only comes when you start thinking about backcasting, which is what

02:21:38.500 do you want your body to function like the day before you take your last breath?

02:21:44.100 Really think about this.

02:21:45.460 So I've taken this to an extreme and people have heard me talk about this in the podcast

02:21:48.660 before, but I come up with this idea of the centenarian Olympics, which is instead of training

02:21:54.280 for like an Olympics that's now where you would actually have to pull vault and run a hundred

02:21:58.740 meters in under 10 seconds.

02:22:00.100 Think about what an Olympic games would look like of a hundred year olds and ask yourself,

02:22:04.920 what do you want to be able to do in that Olympic game?

02:22:07.940 So I have like 18 things I want to be able to do when I'm a hundred, if I get to be a

02:22:11.440 hundred and they're all quite mundane, but they're all very pragmatic and practical.

02:22:16.180 One of them is being able to do a 30 pound goblet squat, which again, I could do today blindfolded

02:22:21.240 on one leg a hundred thousand times, but at a hundred, that's actually going to be quite

02:22:26.000 a task.

02:22:26.720 And it's important because that replicates picking up a toddler, which presumably if I'm

02:22:31.500 a hundred, I've got great grandkids.

02:22:32.940 I want to be able to pick them up.

02:22:34.120 I want to be able to stand up off the floor using no support and only a single point of

02:22:39.780 my own, meaning one arm and both my legs and get up off the floor without help.

02:22:44.620 Can I do that today?

02:22:45.500 Of course.

02:22:46.240 Can you do that today with your eyes shut?

02:22:48.700 Very few hundred year olds can do that.

02:22:50.480 So I've kind of 18 of these things I want to be able to do.

02:22:54.280 And that is backcasting.

02:22:56.120 It's going to the finish line and saying, if I can do that at a hundred, what was I able

02:23:00.920 to do at 90?

02:23:02.180 If I can do that at 90, what must've been true at 60 and before you know it, I realized,

02:23:07.700 wow, if I want to do that at a hundred, now that I'm 46, I actually have to be able to

02:23:11.620 do all of these other things.

02:23:12.540 So there's this whole matrix that comes out of it.

02:23:15.100 That's why that concept for me just resonated so much.

02:23:17.720 Cause I really, I don't think medicine spends enough time backcasting.

02:23:21.420 Yeah.

02:23:21.620 So let me try to add another benefit of backcasting.

02:23:24.140 So just to be clear, backcasting is, so when we forecast what we think is there's a goal

02:23:29.720 that I would like to reach, how do I get there?

02:23:32.000 And in general, what you're thinking are, what are the things that I'm doing in order

02:23:34.780 to get to this goal?

02:23:36.100 When you backcast, what you say is it's the day afterwards.

02:23:39.560 So I have some goal I'd like to be able to do X in a year.

02:23:43.360 And now it's a year and a day and I achieve my goal.

02:23:46.680 And what you ask now is you look back and you say, how did I actually achieve this?

02:23:52.520 And you can also do something called a premortem.

02:23:55.060 So a premortem would be, I have this goal that I'd like to achieve.

02:23:58.520 I'd like to achieve it in a year.

02:24:00.480 It's now a year and a day and I failed.

02:24:02.420 Why did I fail?

02:24:03.060 Why did I fail?

02:24:04.040 Now, here's an added benefit that I would like to offer to you.

02:24:07.680 When we forecast, when we think about, here's my goal, how do I get there?

02:24:12.740 We're thinking about the things we're doing.

02:24:15.300 Here are the decisions that I make along the way.

02:24:17.860 So we're thinking about all the things that are within our decision-making power, within

02:24:21.560 our control, that are in the sort of skill side of the equation.

02:24:24.880 But what happens when you backcast and you say, I actually succeeded, how did I get here?

02:24:30.320 But particularly, what happens when you do a premortem and you say, I failed, how did

02:24:34.560 I get here?

02:24:35.100 Now, you allow luck into the equation.

02:24:39.400 So as an example, if I'm thinking about when I'm 100, I would like to be able to do a 30

02:24:46.920 pound goblet squat, which I've never heard before, but now I'll say that because I learned

02:24:51.300 that.

02:24:52.080 But I'm now 100.

02:24:53.480 It's the day I've turned 100 and I cannot do that thing.

02:24:56.220 Why?

02:24:57.160 Well, I can think of some things like I didn't actually exercise enough, but I can also think

02:25:02.280 of I tripped and I fell and I broke my hip.

02:25:05.100 Now, tripped and fell, that can happen to anybody.

02:25:08.220 Probably not a matter of skill.

02:25:10.380 It's just something can happen.

02:25:11.560 Like you could hurt yourself.

02:25:12.820 But the break my hip might have to do with my bone density.

02:25:15.560 That's going to cause you to explore other things that maybe wouldn't be as obvious.

02:25:19.420 Because you saw, nobody's thinking, you don't see that bad luck can occur if you're not

02:25:26.120 working backwards.

02:25:27.060 And then to your point, once you work backwards, you can now ask yourself some questions because

02:25:31.480 you see the luck.

02:25:33.080 You can say, okay, if it's on the good side, can I increase the chances that that thing

02:25:38.800 happens to me?

02:25:39.540 That I see is luck is going to intervene in my way.

02:25:42.400 Can I increase the chances that's going to happen for me?

02:25:45.100 Okay, so you could ask that.

02:25:45.880 But then on the downside, which is really important to explore, let's say the breaking the hip thing,

02:25:50.660 can I decrease the chances that I break my hip?

02:25:53.420 Yes.

02:25:53.800 What are the things that I could do?

02:25:54.960 If the answer is no, then you can say, are there any hedges available to me?

02:26:00.920 Can I hedge against this bad luck thing happening to me?

02:26:04.260 Sometimes the answer to that is also no.

02:26:07.380 Regardless, what you can say is, okay, I've got my hedges set up.

02:26:10.600 I'm doing what I can to decrease the chances that it occurs, but obviously it still could.

02:26:15.160 So let me think now, before that bad luck intervenes, what am I going to do in response?

02:26:20.560 Because if I plan for that in advance, I'm going to be making decisions right now in a

02:26:26.940 calmer state of mind where I'm not pants on fire, freaking out, rather than being reactive

02:26:32.660 to what the world might deliver me at the time.

02:26:35.180 So if I've already thought about it, if I already have a plan in place, I'm not wasting

02:26:39.560 time.

02:26:40.100 I'm going to be much more calm.

02:26:41.660 I'm going to think about it.

02:26:42.500 So this is one of the big difference between working forwards and working backwards is that

02:26:45.720 when we work forwards and we're thinking about how do we execute on any kind of strategic

02:26:48.960 plan, because we can think about, I want to be able to do this thing when I'm 100 and we're

02:26:53.220 developing a strategic plan for how we get there.

02:26:55.880 We really focus on what are the things that we're going to do that are going to get us

02:26:59.980 there.

02:27:00.280 And we don't say, what are the ways that luck might intervene?

02:27:04.560 But when you work backwards, you naturally start to see the luck.

02:27:07.680 And then you can start to explore all of these questions that then makes your strategic

02:27:11.520 plan much better.

02:27:12.600 It makes you explore different stuff.

02:27:14.700 And does your workbook cover exactly this type of decision tree that we just went through?

02:27:19.060 When is this coming out?

02:27:19.940 A year from now.

02:27:21.180 So I'm just finishing up the draft now.

02:27:23.940 And then you have to lay it out.

02:27:25.660 And you know how that goes with books.

02:27:26.740 Yeah, yeah.

02:27:27.180 So basically what I'm doing with the workbook is I'm taking these concepts that I talk about

02:27:31.400 broadly and sort of bigger idea way and thinking in bets.

02:27:35.380 And I'm saying, OK, how do you practically make a decision?

02:27:39.080 How do you build out a decision tree?

02:27:40.640 How do you think about the way that luck might intervene?

02:27:44.280 How do you think about your own decision making?

02:27:46.100 How do you see the luck?

02:27:47.180 How do you communicate with other people such that you're making sure that you're getting

02:27:52.680 the purest feedback from them, that you're not biasing their feedback, that you're actually

02:27:58.400 hungry for the information and you're seeking the feedback, which is, by the way, the first

02:28:02.060 step, which most of us don't do.

02:28:03.760 Most of us aren't seeking the feedback.

02:28:05.680 So when you're hungry for the feedback, you're actually thinking about what does the future

02:28:10.000 look like?

02:28:10.640 What could the outcomes be?

02:28:11.760 How am I going to react to those things?

02:28:13.880 How do I understand the sum of my experience and what that's supposed to teach me?

02:28:18.460 So it was really trying to think about how do I allow people who don't do this for a

02:28:22.980 living, who don't spend their time building out probabilistic decision trees, and this

02:28:28.040 is not the way that they look at the world.

02:28:30.020 How do I take somebody like that and show them the value and teach them how to actually

02:28:34.240 think within this framework?

02:28:35.600 Well, I think that means we're going to have to sit down and talk in about a year.

02:28:38.820 Yeah, I hope so.

02:28:39.580 When that book comes out.

02:28:40.000 I want to also just say one thing just for the listeners, because I would like you to

02:28:43.660 put it in the show notes.

02:28:44.480 So I'm going to say it right here.

02:28:45.380 What are you doing?

02:28:46.000 A lot of the stuff that we've talked about with luck and skill, I really want to recommend.

02:28:50.000 Well, first of all, on the forecasting stuff, let me just, again, recommend Super Forecasting,

02:28:53.980 which I know is going to end up in the show notes.

02:28:55.480 But there's a really, really wonderful book by Michael Malbison that's called The Success Equation.

02:29:01.640 And it's all about the influence of luck and skill in our lives.

02:29:04.460 It's got a lot of sports stuff in it, but just a lot of really digging down and drilling

02:29:09.660 down into understanding luck and skill and how you sort of understand what the difference

02:29:15.280 is between the range of activities and sort of where you can see more influence of skill,

02:29:19.460 where you can see more influence of luck, how you can sort of parse those apart.

02:29:23.660 This huge influence on our lives is luck.

02:29:27.280 And he really drills down into that in a way that if people are really interested in the

02:29:32.140 conversation that we had today, then I guarantee they would be really interested in the success

02:29:37.520 equation.

02:29:38.660 And Malbison also, I mean, you can just search him and you can see a whole bunch of videos of his

02:29:43.020 where he talks about luck and skill.

02:29:45.860 A lot of his writing talks about luck and skill and sort of separating the two and understanding

02:29:50.580 the different influences.

02:29:51.840 And yeah, I just can't recommend that book more.

02:29:54.460 We'll put together a pretty robust set of notes here, as we always do.

02:29:57.340 And it will point to everything we've spoken about.

02:29:59.660 Plus, I'm sure after the fact over the next couple of weeks, you and I will have back and

02:30:03.640 you'll think of something else that you want to hear.

02:30:05.460 So I'd love to make this the sort of the compendium for all these things.

02:30:08.180 And then, of course, I can't wait to sort of talk about this workbook.

02:30:10.340 It sounds like it's kind of an amazing tool.

02:30:12.700 I mean, again, I would say like in the last few hours, I've realized that I'm not nearly

02:30:16.980 as good a decision maker as even I think I am.

02:30:19.020 And I don't say that to be sort of arrogant.

02:30:20.820 I just think, look, I understand decisions at least as well as anybody else has been my bias.

02:30:26.500 And I'm at least as probabilistically savvy.

02:30:29.260 But I realize like there's tons of blind spots that I'm still experiencing.

02:30:32.900 And I'm not even close to approaching the potential that that a person would have to

02:30:38.180 make better decisions.

02:30:39.060 So that said, I find myself somewhat overwhelmed by how complicated this is.

02:30:44.840 I'm already thinking like Travis is going to put together some great

02:30:47.860 tear outs that will hold us over until your book comes out.

02:30:50.860 But having a book that could be sort of a guidepost to a decision matrix would be fantastic.

02:30:57.100 And let me just for people who have listened to this and felt sort of overwhelmed.

02:31:00.800 It's funny because this happens to be the chapter that I'm writing right now is that it's really,

02:31:06.280 really good to have this framework to be thinking about decisions and to really recognize the

02:31:11.240 probabilistic nature of the world and the way that things turn out for you and understanding

02:31:16.380 what the structure of a really good decision is.

02:31:18.600 That being said, that doesn't mean that you have to sit down with a slide rule for every

02:31:23.700 decision that you make.

02:31:24.640 And in fact, for the majority of decisions that you make in your life, you can actually

02:31:29.000 decide quite fast.

02:31:30.600 And what's interesting is that the way to really understand when can I decide fast is to understand

02:31:38.000 this broader framework.

02:31:39.220 This broader framework that would allow you to take the bigger decisions in your life and

02:31:42.700 actually start to think about them in this way where you're really forecasting them is

02:31:46.560 actually the way to understand when can I go really fast.

02:31:50.080 And what it will allow you to do is stop taking 15 minutes to decide what to order in a restaurant

02:31:56.180 because you have to understand this framework in order to sort of get when you can go fast

02:32:00.900 and when you can go slow.

02:32:02.060 Because when I do talk to people about this kind of framework, they do feel a little bit

02:32:05.820 overwhelmed and they're like, oh my gosh, how am I ever going to make a decision again?

02:32:09.960 Like I'm going to need a supercomputer in order to do it.

02:32:12.720 It's very few decisions that you actually need to take a ton of time on.

02:32:15.980 Mostly you can decide fast.

02:32:17.900 And on the ones that are bigger decisions, embracing the probabilistic nature actually

02:32:24.460 gets you out of the analysis paralysis.

02:32:26.440 Because what you realize is once you've identified your options and you've got an option that you

02:32:30.400 see is better relative to the other ones, you stop focusing on achieving absolute certainty

02:32:36.000 and start focusing on achieving relative to the other things that I could do.

02:32:40.260 This one definitely looks better to me.

02:32:42.440 And because I recognize that there's all sorts of stuff I don't know and there's all sorts

02:32:46.980 of different ways the world can turn out.

02:32:48.700 And as long as I'm taking the things that occur in my life and trying to learn from them

02:32:52.920 in a real way, I'm not going to be so terrified of sort of the downside.

02:32:56.840 And that really frees you up.

02:32:58.420 You've said that so, so elegantly.

02:33:00.120 And I think that reiterates this sort of the notion that intuitively people get it, right?

02:33:05.300 You go fast on the straightaway and slow on the curves.

02:33:07.420 This framework shows you where the curves are.

02:33:09.380 Yeah.

02:33:09.620 And that frees you up to go faster on the straightaways.

02:33:11.720 Yes.

02:33:12.100 And interestingly enough, it frees you up relative to what you were doing before to go faster

02:33:18.220 on the curves as well.

02:33:19.560 Absolutely.

02:33:20.360 To not try to figure out in that moment, if I don't know the exact right path to take,

02:33:26.380 I'm paralyzed and I just need to slam the brakes on.

02:33:28.940 But to recognize when, well, this path is going to be good enough.

02:33:32.620 I'll spare all the listeners the race car analogy that is like zooming into my cortex

02:33:37.660 at the moment because there is a perfect analogy to driving around a corner in a race car,

02:33:42.900 but I will spare everybody that.

02:33:44.040 You can put it in the show notes.

02:33:45.080 There we go.

02:33:45.940 Well, Annie, thanks so much.

02:33:46.720 This was really great.

02:33:47.500 I really appreciate you responding to my random request to sit down and talk about this.

02:33:52.640 And I can't wait to do it again when the new book comes out.

02:33:54.180 Well, thank you.

02:33:54.960 I'm excited to come back when the workbook is done.

02:33:56.940 But this was like an amazing conversation, very different than most of the podcasts I've

02:34:00.740 done.

02:34:00.960 So I really appreciate it.

02:34:02.160 That's the way we do things here.

02:34:03.580 We go deep.

02:34:04.580 Yeah.

02:34:04.820 You can find all of this information and more at peteratiamd.com forward slash podcast.

02:34:12.420 There you'll find the show notes, readings, and links related to this episode.

02:34:16.540 You can also find my blog at peteratiamd.com.

02:34:20.040 Maybe the simplest thing to do is to sign up for my subjectively non-lame once a week email

02:34:24.520 where I'll update you on what I've been up to, the most interesting papers I've read,

02:34:28.200 and all things related to longevity, science, performance, sleep, et cetera.

02:34:32.980 On social, you can find me on Twitter, Instagram, and Facebook, all with the ID Peter Atiyah

02:34:37.760 MD.

02:34:38.420 But usually Twitter is the best way to reach me to share your questions and comments.

02:34:42.020 Now for the obligatory disclaimer.

02:34:43.660 This podcast is for general informational purposes only and does not constitute the practice

02:34:47.820 of medicine, nursing, or other professional healthcare services, including the giving of

02:34:52.700 medical advice.

02:34:53.420 And note, no doctor-patient relationship is formed.

02:34:56.880 The use of this information and the materials linked to the podcast is at the user's own risk.

02:35:01.960 The content of this podcast is not intended to be a substitute for professional medical

02:35:05.600 advice, diagnoses, or treatment.

02:35:08.200 Users should not disregard or delay in obtaining medical advice for any medical condition they

02:35:12.460 have and should seek the assistance of their healthcare professionals for any such conditions.

02:35:17.920 Lastly, and perhaps most importantly, I take conflicts of interest very seriously for all of my

02:35:22.620 disclosures, the companies I invest in and or advise, please visit peteratiyahmd.com forward

02:35:29.320 slash about.

The Peter Attia Drive - July 01, 2019

#60 - Annie Duke, decision strategist： Poker as a model system for life—how to improve decision making, use frameworks for learning, and apply 'backcasting' to boost your odds for future success

Episode Stats

Summary

Transcript