The Peter Attia Drive - #60 - Annie Duke, decision strategist： Poker as a model system for life—how to improve decision making, use frameworks for learning, and apply 'backcasting' to boost your odds for future success

00:00:00.000 Hey everyone, welcome to the Peter Atiyah Drive. I'm your host, Peter Atiyah.

00:00:10.140 The drive is a result of my hunger for optimizing performance, health, longevity, critical thinking,

00:00:15.600 along with a few other obsessions along the way. I've spent the last several years working with

00:00:19.840 some of the most successful top performing individuals in the world. And this podcast

00:00:23.600 is my attempt to synthesize what I've learned along the way to help you live a higher quality,

00:00:28.360 more fulfilling life. If you enjoy this podcast, you can find more information on today's episode

00:00:33.000 and other topics at peteratiyahmd.com.

00:00:41.440 Hey everybody, welcome to this week's episode of the drive. I'd like to take a couple of minutes

00:00:46.000 to talk about why we don't run ads on this podcast and why instead we've chosen to rely entirely on

00:00:51.660 listener support. If you're listening to this, you probably already know, but the two things I care

00:00:56.480 most about professionally are how to live longer and how to live better. I have a complete fascination

00:01:02.780 and obsession with this topic. I practice it professionally and I've seen firsthand how

00:01:07.480 access to information is basically all people need to make better decisions and improve the

00:01:12.120 quality of their lives. Curating and sharing this knowledge is not easy. And even before starting the

00:01:17.600 podcast, that became clear to me. The sheer volume of material published in this space is overwhelming.

00:01:22.820 I'm fortunate to have a great team that helps me continue learning and sharing this information

00:01:27.880 with you. To take one example, our show notes are in a league of their own. In fact, we now have a

00:01:33.840 full-time person that is dedicated to producing those and the feedback has mirrored this. So all of this

00:01:39.720 raises a natural question. How will we continue to fund the work necessary to support this? As you

00:01:45.980 probably know, the tried and true way to do this is to sell ads. But after a lot of contemplation,

00:01:51.100 that model just doesn't feel right to me for a few reasons. Now the first and most important of

00:01:56.300 these is trust. I'm not sure how you could trust me if I'm telling you about something when you know

00:02:02.100 I'm being paid by the company that makes it to tell you about it. Another reason selling ads doesn't

00:02:07.580 feel right to me is because I just know myself. I have a really hard time advocating for something

00:02:12.840 that I'm not absolutely nuts for. So if I don't feel that way about something, I don't know how I can

00:02:18.500 talk about it enthusiastically. So instead of selling ads, I've chosen to do what a handful

00:02:23.280 of others have proved can work over time. And that is to create a subscriber support model for my

00:02:29.320 audience. This keeps my relationship with you both simple and honest. If you value what I'm doing,

00:02:36.260 you can become a member and support us at whatever level works for you. In exchange, you'll get the

00:02:41.840 benefits above and beyond what's available for free. It's that simple. It's my goal to ensure that no

00:02:47.860 matter what level you choose to support us at, you will get back more than you give.

00:02:52.960 So for example, members will receive full access to the exclusive show notes, including other things

00:03:00.380 that we plan to build upon, such as the downloadable transcripts for each episode. These are useful

00:03:06.760 beyond just the podcast, especially given the technical nature of many of our shows. Members also get

00:03:12.900 exclusive access to listen to and participate in the regular ask me anything episodes. That means

00:03:19.840 asking questions directly into the AMA portal and also getting to hear these podcasts when they come

00:03:25.800 out. Lastly, and this is something I'm really excited about. I want my supporters to get the best

00:03:30.780 deals possible on the products that I love. And as I said, we're not taking ad dollars from anyone,

00:03:35.620 but instead what I'd like to do is work with companies who make the products that I already love and would

00:03:40.600 already talk about for free and have them pass savings on to you. Again, the podcast will remain

00:03:47.740 free to all, but my hope is that many of you will find enough value in one, the podcast itself, and two,

00:03:56.260 the additional content exclusive for members to support us at a level that makes sense for you.

00:04:01.680 I want to thank you for taking a moment to listen to this. If you learn from and find value in the

00:04:06.180 content I produce, please consider supporting us directly by signing up for a monthly subscription.

00:04:11.700 My guest this week is Annie Duke. Annie is the author of the bestselling book,

00:04:16.280 Thinking in Bets, Making Smarter Decisions When You Don't Have All the Facts. And some of you may

00:04:20.580 recall a few months ago, I talked about this book a little bit in my Sunday morning newsletter because

00:04:26.620 it was just one of those books where even halfway through it, I realized this was a book everybody

00:04:30.740 needed to read and I couldn't get enough of it. And I immediately reached out to Annie and said,

00:04:34.400 I'd love to have you on the podcast and what follows is that discussion. But for those of

00:04:38.980 you who may have not got that or missed that, Annie was one of the top poker players in the world.

00:04:43.600 She played professionally for about 20 years. In 2004, she won her first world series of poker

00:04:48.700 bracelet. That same year, she won $2 million in a winner take all invitation only world series

00:04:53.860 poker tournament of champions. In 2010, she won the prestigious NBC national heads up poker

00:04:59.020 championship. Prior to all of that, by the way, and we talk a little bit about this,

00:05:03.280 she was actually awarded a National Science Foundation fellowship to study cognitive psychology

00:05:07.740 at the University of Pennsylvania. So poker actually is something that kind of came later

00:05:11.180 to her in life. She now spends her time writing, coaching, and speaking on a wide range of topics

00:05:16.360 from decision fitness, emotional control, productive decision groups, embracing uncertainty.

00:05:21.040 She's also done a lot of work in charity. And unfortunately, we don't get into any of that. I had planned

00:05:26.000 to, but as is often the case, we get so far into something in this case, obviously poker and

00:05:31.840 uncertainty and decision-making that after three hours or so, we hadn't even got to some of the

00:05:35.900 other things that she's done. We talk a lot about how she got into this. And of course, we do spend

00:05:41.080 some time defining poker because if you're not a poker player, and I'm not a poker player, I mean,

00:05:44.680 I sort of know the rules, but I think it is important to understand the nuances of the game,

00:05:50.440 at least as far as the rules go and the different types of poker. So even if you've never played a hand

00:05:56.520 of poker in your life and don't know the difference between Texas Hold'em and Blackjack, for that

00:06:00.960 matter, you'll still get a ton out of this because we really explain that stuff. And we contrast poker

00:06:06.280 with other things like chess, where clearly you have to have a lot of skill to play them,

00:06:12.520 but luck is not an element. We talk about why poker is in some ways the model system for decision

00:06:18.980 making in the real world. We talk about backcasting, which is something that I believe I alluded to in

00:06:26.620 my newsletter that is what really got me excited about the way she described this. And we talk about

00:06:32.160 doing pre-mortem exams versus just post-mortem exams. We go through the decision matrix, meaning

00:06:38.900 how do you look and evaluate decisions after the fact when you have to be critical of both the

00:06:44.700 decision making process and the outcome. And I have to tell you, this is just yet another example

00:06:49.820 of a podcast where I came away learning so much. And in preparing for this, I obviously have read

00:06:56.440 the books and I've thought a lot about decision making and I know quite a bit about probability

00:06:59.940 theory. It didn't matter. Annie still took my understanding of this to another level. And in

00:07:05.780 the process, I realized she is working on another book, which is a workbook to help with decision

00:07:09.860 making. That's going to be out next year. So one, A, I can't wait to get that. And two,

00:07:14.080 we're going to have to have Annie back when that book comes out and do a real deep dive

00:07:18.080 on how to implement that workbook. So I could go on for a lot on this, but you don't want to

00:07:22.800 hear me talk. You want to hear Annie talk. So without further delay, here is my guest, Annie Duke.

00:07:30.620 Annie, thanks so much for coming all the way up to New York. I'd like to say just to see me,

00:07:34.760 but now you've let the cat out of the bag that I'm the second excuse to be here today.

00:07:38.800 Yes. When I get requests, I try to pack them into the same day for efficiency purposes.

00:07:43.400 Are you from Philly originally?

00:07:45.180 I'm not from Philly, but my father is from Philly originally. So yeah, my dad grew up in

00:07:50.320 Philadelphia and graduated from West Philly High actually. So he's born and bred and I live in

00:07:56.880 Philadelphia now. It's actually my third time through. So there's clearly some kind of magnetic

00:08:02.240 center to Philadelphia for me.

00:08:04.140 Are you an Eagles fan?

00:08:05.200 I am.

00:08:05.660 I'm sorry. I can't. I'm so angry about them beating the Patriots two years ago.

00:08:10.180 I'm just so upset about it.

00:08:11.300 Oh, please. Oh, I'm so sorry for you that they didn't get their sixth Super Bowl that day and

00:08:17.120 they had to wait two more years.

00:08:19.140 I don't know. It just bothered me.

00:08:20.460 All right. So wait, are you a Patriots fan?

00:08:23.080 I am.

00:08:23.400 Okay. So interestingly enough, I am a Red Sox fan because I grew up in New England and

00:08:28.280 it's kind of just an accident of sort of the timing of my own life that I'm an Eagles fan,

00:08:34.300 which has to do with, I was never much interested in football when I was growing up, which isn't

00:08:39.300 that crazy for when I was growing up in terms of the Patriots because they were awful.

00:08:43.120 Horrible.

00:08:43.240 Oh, just horrible. It was horrible. So I liked the Celtics because Larry Bird and Kevin McHale

00:08:49.680 and obviously the Red Sox. It was like Fred Lynn and Carl Yastrzemski and Carlton Fisk and like that

00:08:55.600 amazing team. So, and we used to go like once a year. So I would like to remind you as you're so

00:09:02.040 upset that they didn't get number six. On that day.

00:09:05.620 Of what it was like to be a Red Sox fan.

00:09:08.240 I know. I know. Well, you brought up Fisk, right? So it's like, yeah.

00:09:11.060 Was it the 79 team or the 76 team?

00:09:14.240 I think it was 76, but someone's going to correct me, but yeah.

00:09:16.920 Where it's like almost.

00:09:18.360 No, I know. That's, it's hard to watch. You know what it is? I'm a Belichick fan. So people sort of

00:09:22.740 say, oh, you're a Patriots fan. Like wow, wow, wow. But it's not that I disproportionately like anyone

00:09:27.480 on that team. Although I think Tom Brady is, I think unequivocally the greatest quarterback of all

00:09:32.240 time at this point. But to me, it's just, it's Belichick. Like I'm obsessed with the way this guy

00:09:38.900 operates. Like you always get asked the question, like if you could meet anybody famous, who would

00:09:42.640 it be? I mean, at the top of my list would have to be him. Like, and again, it wouldn't be in the

00:09:47.900 setting of him giving glib answers in a news conference. Cause that wouldn't be that much

00:09:51.640 fun. Although it's funny as hell to watch. If I could spend a day on his boat with him,

00:09:56.460 really understanding the discipline that he brings to what he does, it would just,

00:10:00.300 it would just amaze me. There's a beautiful article that was written probably five years ago.

00:10:05.000 It was just told in a really funny way. Cause it was the gist of the article was everybody's got a

00:10:09.340 Belichick story and it's players, coaches, all telling these stories of him that really highlight

00:10:16.700 this single minded focus of winning. That is kind of amazing to me, even though from a life ethos

00:10:23.140 standpoint, I don't really care that much about sports anymore. I've sort of, there was a day when

00:10:27.180 it really upset me if my team lost. Now it's sort of, oh, that sucks.

00:10:30.620 Well, it's good that you've managed to come to that point.

00:10:34.540 Oh my God. Like I could tell you stories about when I was in high school, if my team lost,

00:10:38.320 it was like, if you want to get at least one degree, just one degree from Bill Belichick,

00:10:42.720 you should have Michael Mbardi on who wrote Gridiron Genius. It's a great book about leadership.

00:10:48.680 He worked with Bill Walsh, he worked with Belichick. He really kind of talks about what the

00:10:54.660 leadership qualities of those two particular coaches. He's really great. You should look him up.

00:11:00.200 Do you know him?

00:11:00.620 I do, yes.

00:11:01.140 Well, then I'm going to ask you to make that introduction.

00:11:02.980 I will make that introduction.

00:11:04.200 I met Bill Walsh about a month before he died and it was the most sort of simple, he was at

00:11:10.700 Stanford and I guess he was getting some medical care and I was there. I lived in the Bay Area at

00:11:15.620 the time, but I still swam at the Stanford pool. And one day I was just finishing a swim and I was

00:11:20.500 walking back to the parking lot and I can't believe it, but Bill Walsh is like walking alone,

00:11:25.060 like on this part of the campus. I normally would never interrupt somebody to be the fan boy

00:11:29.780 because I just feel like it's sort of, I don't know, that's not my style to do it,

00:11:32.940 but I couldn't help it. And I was, oh my God, I just want to shake your hand. And of course,

00:11:37.860 like a total idiot, I just started telling him all of my favorite moments of his coaching career

00:11:42.900 as though he doesn't remember them. Like, do you remember the drive at the end of the second

00:11:48.780 Bengals Super Bowl when there were only two minutes and four seconds left? And remember when Taylor

00:11:53.620 and he was just the kindest, he totally entertained all of my nonsense. Like, yeah,

00:11:59.040 those were some great times. Those are some great memories. And sadly he died like a month later.

00:12:03.580 I mean, he was sick in that moment. He was totally able to, anyway. So, but that said,

00:12:07.400 I'm not happy about the Eagles and we'll just let that slide.

00:12:11.420 Literally, it was like two years later. Here's the thing. Belichick is in no danger of not getting

00:12:16.840 into the Hall of Fame. Now there's Tom Brady. This is what I'm trying to remind you of the Red

00:12:23.000 Sox, letting the Eagles have their one moment. Yeah. Well, now the Red Sox can't stop doing it.

00:12:27.960 It's also sort of. Yeah. Cause I'd like to remind you that the Eagles did get their first try in like

00:12:33.120 ever at the Super Bowl against the Patriots in the first place in loss. So like, come on,

00:12:38.520 it's like crumbs, crumbs for the peasants.

00:12:42.720 So first of all, I read your book. I don't know how long ago it's been, but I mean, it was

00:12:46.280 probably 10 minutes before I reached out to you on Twitter. I loved it. And I think we all sort of

00:12:50.940 read books and apply our own filter to them. And of course, one of the filters is on this business

00:12:56.160 of decision-making and in medicine, we make decisions all the time and we make decisions

00:13:01.560 within complete information. And I remember explaining to somebody once something that I

00:13:06.180 think I either heard you talk about on a podcast after the fact and describe it more eloquently.

00:13:10.340 And I think you also alluded to it in the book, which is people always think of chess as this perfect

00:13:15.460 game for decision-making. So actually my wife, I was having this discussion with, I said, no,

00:13:19.820 actually chess is not a great example because everybody has perfect knowledge at every moment

00:13:23.820 in the game. So it's not to discount the importance of strategy in chess. People who are brilliant at

00:13:29.080 chess are brilliant, but it's not the perfect analog for decision-making in life. For that,

00:13:34.600 poker is much better because you have incomplete information and that tends to mimic what we see in

00:13:40.340 life. And in medicine, I think you almost never have complete information. And that's sort of one

00:13:45.600 of the things that is both frustrating because how can you make a serious decision when you don't know

00:13:51.180 all of the facts, but at the same time, I think makes it intriguing and interesting and gives you

00:13:56.520 this sense of there's a craft that can be mastered here that is less sort of amenable to robotic

00:14:02.340 decision-making.

00:14:03.120 Tess is missing this particular element, which decision-making in general has,

00:14:08.100 which has to do with incomplete information. So in chess, I can see the player's whole position.

00:14:13.720 And what that means is that I should be able to calculate out if I'm considering a move,

00:14:18.060 I know what all the possible moves are that the player can make. And then I can think of what

00:14:21.940 all the possible moves are that I can make in response and so on and so forth, so that I should

00:14:25.800 be able to calculate out with absolute perfect knowledge here, what the right move is at any given

00:14:30.640 time. It's missing another element though, that I think is really important to think about,

00:14:35.060 which is a very strong influence of luck. So in chess, the pieces stay where they are. Nobody

00:14:42.300 takes dice, you roll an 11, and two pawns come off the board, or if it's snake guys, checkmate.

00:14:49.020 Or maybe something good happens, you roll a six and you get an extra queen. That doesn't happen in

00:14:55.140 chess so that there's this, we know that the pieces are only ever going to move through an act of skill.

00:14:59.940 And this idea of the strong influence of luck is really, really important. Meaning, if I were to

00:15:06.940 do the same thing in the same situation, and then post decision, are there other things that could

00:15:13.200 reasonably happen that I don't have control over? I don't have control over which thing is actually

00:15:17.860 going to occur. It's actually a really important element to the way we make decisions. If you think

00:15:22.340 about it in poker, if I put my money in with aces, and you put your money in with fives, you know,

00:15:28.820 80 ish percent to win there, slightly better. That means 80% of the time I'm going to win. But that

00:15:34.420 doesn't mean that you can't win 20% of the time you can win. And I have no control either way. I don't

00:15:39.900 know if on that particular iteration, I'm going to see the 80% or the 20%. That really crosses all

00:15:46.140 decision making. So you can tell me this would be my intuition as not a medical practitioner. But

00:15:52.140 not only are you dealing with incomplete information in medicine, but even if you did have

00:15:57.380 complete information, if you apply a treatment in a situation, and it happens to work, there's all

00:16:03.840 sorts of stuff going on, I assume that's relatively stochastic, that we don't really understand,

00:16:08.660 where if we were to do that same thing again, it may not work in the next situation. So we always

00:16:13.720 want to think about that, because that has a very big effect on our decision making.

00:16:17.360 Yeah, you're absolutely correct. And just to hear you explain it that way, that might even be the

00:16:21.660 more important factor, actually, than the absence of complete information. Now, the definition of a

00:16:26.880 chaotic system versus just a stochastic system is even more profound, where even a slight change in

00:16:31.980 the initial condition can produce a profound change in the outcome. We don't even have to posit that we

00:16:37.460 can just leave it at the same initial conditions can produce different outcomes, right? We can kind of

00:16:42.920 unpeel it in this particular way. Let's say that we're dealing in a world where we had perfect

00:16:47.920 information. I think it's important to think about this to understand why chess is really so bad in

00:16:53.360 terms of thinking about these kinds of decisions. It's an interesting decision making problem. I'm not

00:16:58.080 trying to slam chess, I think it's a great game. And it teaches critical thinking skills. And it teaches

00:17:02.620 you to think many moves ahead and to think about what other people know. And there's all sorts of great

00:17:07.520 benefits that it has, it just happens to be missing a couple things that I think are really

00:17:11.040 important. But let's say that we had perfect information. For example, if I have a coin,

00:17:16.480 and I've weighed it, so that I know that the weight is correct, I've examined it, and I can see that it's

00:17:24.260 two sided, then I now have perfect information. I know exactly what I need to know about the coin

00:17:31.260 to understand how often it's going to flip heads and how often it's going to land on tails. But what I

00:17:37.180 can't know is what it will land on the next try. So that's that issue of luck. So if you flip and

00:17:43.720 it lands heads, it's reasonable for me to say it could have landed tails. And if we did that again,

00:17:50.080 that tails is totally a possibility, regardless of what I happen to have seen on this particular

00:17:54.360 flip. So and then what we can do is say, okay, so even in conditions of perfect information, we don't

00:18:01.180 really know which outcome we're going to get, we might know the frequency, how often that might occur,

00:18:06.180 or how often we could expect it to occur, but we don't know for sure what will happen this time.

00:18:10.700 And then you can take it further and say, but what if you haven't examined the coin?

00:18:14.700 And based on some set of outcomes, you're now trying to derive what the coin looks like.

00:18:20.220 Now that becomes even harder. And that's where you get into something that looks very little like

00:18:25.920 chess. Why is it do you think that as humans, we, at least most people, most of us don't seem

00:18:33.460 innately wired to understand probability theory because I studied math as an undergrad and I

00:18:39.360 spent so much time in it. I think I take for granted now how easy it is for me to think

00:18:44.840 probabilistically. I sort of view the world as a stochastic map, but I'm pretty sure I wasn't born

00:18:50.240 that way. And I'm pretty sure that nothing, maybe this isn't correct, but I doubt that we were

00:18:55.180 evolutionarily wired to do that. Or I feel like we would see it more often in the way people behave.

00:19:00.600 And I feel like the absence of that in the general person's point of view, who hasn't been forced to

00:19:05.920 be trained in that way, speaks to that. This strikes me as one of the more difficult things in

00:19:10.400 explaining anything. This touches everything from politics to business to science, because it's too

00:19:18.400 easy to say, well, that didn't happen. Therefore that couldn't have happened as opposed to describing

00:19:25.200 it the way you did, which is no, actually it could have happened. And if the same initial

00:19:28.300 circumstance happens again, by the way, this thing could happen.

00:19:30.780 I would imagine if we were to take the distribution of how naturally probabilistic thinking comes to

00:19:38.180 somebody, I'm guessing that you were probably born closer to the right tail. That being said,

00:19:43.820 I guarantee you that if I followed you around for a day, I could find all sorts of spots where you're

00:19:48.740 not thinking probabilistically. And I could in particular find all sorts of spots where your initial gut

00:19:55.380 reaction is not probabilistic and you have to stop yourself.

00:19:58.820 I'm positive. I'll give you an example of a game I played. That's not an exact example of this,

00:20:04.260 but it humbled the hell out of me. And I always go back and play this with other people. So the idea

00:20:10.100 of the game was to teach people what a 95% confidence interval looks like. And this is back when I was at

00:20:15.360 a consulting firm called McKinsey and company. And I was a part of the risk practice. Our whole job was

00:20:19.460 walking around and helping companies manage risk. So we did this at an offsite one day to help us kind of

00:20:26.040 figure this out. So the moderator goes up with 20 questions. Have you played this game?

00:20:30.600 So I have, and this is one of Phil Tetlock's favorite who wrote expert political judgment and super

00:20:36.380 forecasting, which I highly recommend anybody interested in probabilistic thinking.

00:20:40.200 I ended up reading super forecasting years after this experiment.

00:20:43.420 So you know, he's got this in there. Yeah.

00:20:44.960 So I'll tell it just for the listener. So you take, and I still have it, by the way,

00:20:47.680 I still have the 20 questions. I was terrible at this.

00:20:49.960 I'll tell you what my score was after. I'm so pissed about it too. Like how bad I was.

00:20:53.580 Mine was like zero. So go ahead. And I've really tried. It's just like the questions that I got

00:20:59.060 were really out of my knowledge base. And it was like, I was trying to have really wide,

00:21:02.660 but it was a disaster. Okay. So what the moderator does is they give you 20

00:21:06.200 questions. Each one has a quantitative answer that is known. That is known. Yeah. And the question for

00:21:12.980 you, the participant is for each question, give me a bracket that contains the answer. So for example,

00:21:20.540 what is the average height of a male in the United States? An appropriate answer would be

00:21:27.440 somewhere between four foot six and six foot one, because that would be a reasonable guess

00:21:34.380 at a bracket that contains the average height of a male.

00:21:38.040 Right. So the way that I've heard it is that you want the bracket to contain it 95% of the time.

00:21:42.700 Which means after you've done all 20, you should have exactly one incorrect and 19 correct. And the

00:21:51.040 purpose of saying 95% confidence is if they said to you, you have to have a hundred percent

00:21:56.320 confidence. You can sandbag it. One inch tall. Yeah. There's somewhere from one inch tall to a

00:22:00.780 million miles tall. And the questions in fairness to people like you and I who have flunked the test,

00:22:05.340 the questions are hard. So did you get, how many votes did Abraham Lincoln receive? I didn't get that

00:22:10.360 one. I got like, what is the distance between Jupiter and its nearest moon? It's like, I don't

00:22:15.460 have any goddamn clue, but how many people were voting during Lincoln's? I don't know. And I had

00:22:22.520 this really wide range, you know, it's great, but it's such a great exercise because you think, okay,

00:22:28.320 I've got this under control. I'm going to just try my best to get everyone correct with a reasonable

00:22:34.820 guess and I'll probably get one wrong. And I think in the end I got 12 right and eight wrong, which of

00:22:40.740 course is like a 60% confidence interval on things that are perfectly knowable. And so, yes, I completely

00:22:47.820 agree with you. And of course your initial example is a better one, which is how many times do I make

00:22:53.100 either a decision or have a gut reaction that I have to actively talk myself out of?

00:22:59.600 The reason why I bring that up is that I think that there's a couple of things that we want to

00:23:03.220 sort of keep in mind. And then if you want to talk about sort of like from a, like an evolutionary

00:23:07.500 story, why it is that we're not particularly wired to think probabilistically, we can get into that.

00:23:13.180 But what I want to get across is this, that first of all, we're all sort of like sitting,

00:23:17.860 we have proclivities, you know, where we're sitting on a distribution, somewhere on the distribution in

00:23:22.860 terms of sort of how we're born, how we're wired to think probabilistically. And that doesn't mean

00:23:29.660 that we should say either we're there or we're not. I mean, the idea is that wherever we sit on

00:23:35.200 the distribution or whatever our personal distribution looks like, we just want to shift

00:23:39.620 it to the right. So wherever you are, you want to accomplish two things. One is it would be really

00:23:46.240 good if you thought probabilistically a little bit more. Why? Because the world is probabilistic.

00:23:53.980 And so the more accurately you can think about what the outcomes of your decisions might be,

00:23:59.720 because they're not deterministic. When you make a decision, there's many, many things that could

00:24:04.680 occur with some likelihood of each of those things occurring. And understanding that particular fact

00:24:09.040 is incredibly important to good decision making. So the more that you can shift yourself a little bit

00:24:13.260 more on the side of probabilistic thinking, the better off you are, no matter where you started.

00:24:17.260 And then the second thing we want to accomplish, which is what I was trying to get at,

00:24:20.040 at the catching yourself part, is to understand that we are wired this way. So it's really hard

00:24:26.960 at all times to think that you don't know things for certain, to sort of live in this counterfactual

00:24:34.460 world. And it's frankly, kind of impractical, it would be hard to like drive, for example, if we were

00:24:39.460 living counterfactually all the time. So the idea is that when we do make an error to catch it faster,

00:24:45.780 because for most things, when we make an error, and we we have too much confidence, or we're thinking

00:24:52.140 about something as deterministic, when we should be thinking about it as probabilistic, for most

00:24:57.140 things, we don't catch it at all. So if we can just catch it a little bit more often, and if we can

00:25:03.060 catch it a little bit more quickly, we're a lot better off. And these small shifts, if you can get a

00:25:09.560 couple percentage points better, you're way better off in your decision making.

00:25:14.700 So let's go back to kind of where you started thinking about this, which was through the lens

00:25:18.800 of playing competitive poker. How did you end up playing poker?

00:25:22.580 Chaos theory. So let me just say that when I started playing poker, it wasn't all over television,

00:25:27.540 because I think that otherwise, that would be a really simple answer. Like I was in college,

00:25:31.160 and I wanted to make money. And I saw poker on TV, and I started playing online,

00:25:35.860 I think would be the origin story of the majority of poker players today. Not my origin story,

00:25:41.760 because I'm way too old for that.

00:25:43.300 Where did you go to college?

00:25:44.140 Columbia.

00:25:44.620 Okay. Yeah.

00:25:45.320 So you're here in the city.

00:25:46.340 So I'm here in the city, and there's no internet. I mean, I think there's some sort of, I don't know,

00:25:51.720 Al Gore was inventing it, or something. I don't know, but it didn't exist. So the first thing that

00:25:56.740 happened on my way to becoming a professional poker player is that my brother, when he was in high

00:26:02.940 school, got very, very interested in chess, started really studying it, became incredibly good at the

00:26:08.560 game. He got accepted to Columbia, decided to take a year deferment to live in New York and study with

00:26:16.680 a grandmaster. During that year, he started playing poker. All sorts of things happened. He lost his

00:26:23.140 college nest egg. He eventually did very well. But in this particular year, he was learning,

00:26:29.760 let's say. And he was learning maybe on the money that had been saved for college. It was a little

00:26:34.640 bit of money. It wasn't a lot. Anyway, he had started playing. And then I followed to New York

00:26:39.640 to go to Columbia a year later. I actually went. I didn't defer. And by the end of that year, my

00:26:45.040 brother was very good. So now we're both living in New York at the same time. He actually doesn't

00:26:49.940 end up going much later. He ended up going, I think, for like six months, but he ends up never

00:26:54.280 finishing college. And he's playing poker professionally. And I'm going and watching

00:26:59.200 him sometimes. So I'm just sort of sitting behind him. And what does that mean at this

00:27:02.520 era of time? These are like smoky rooms in New York City? Yeah. There was a place called the Mayfair,

00:27:08.120 which had a game. And they had poker and they had backgammon. Before that, my brother was playing

00:27:15.920 at a place called the Bar Point. For people who don't know, that's a backgammon term. And it was like

00:27:21.940 a backgammon club. People also played gin there. It was on 14th and 6th. And in the back room,

00:27:28.940 there was a poker game. This was before I came around. He was sort of already out of this place

00:27:33.060 by the time that I came around. Although I did spend a summer here where I got to see him sort

00:27:37.500 of playing in that particular place. So he found out about that through like his interest in chess,

00:27:43.420 ends up there in the back room. There's a game that starts on Friday night and ends on Monday morning.

00:27:48.000 And by that, I mean, that's how long the people played. They didn't get up and leave and come

00:27:53.880 back. We can have a whole other discussion about sleep deprivation and decision making,

00:27:57.880 but we'll save that for another time. Yeah, this was in the early 80s. So I believe there was

00:28:01.820 cocaine involved. Yeah, I'm sure. But at any rate, so he starts playing in that game. And that's

00:28:07.080 actually where he... So he cuts his teeth. That's where he's cutting his teeth. That's where he's sort of

00:28:10.640 learning how to play. That's where he's losing this college money. And eventually, so he's playing

00:28:16.620 in that game. He's not doing very well. And he starts figuring out, oh, actually, it's sort of in

00:28:22.080 the same way he approached chess. He sort of found whatever books he could find. And he found a few

00:28:26.960 other people who were kind of really interested in learning the game. And he sort of starts working

00:28:30.940 the game. And eventually, he basically sort of breaks that game. Like he becomes so good

00:28:37.840 that I can't remember whether they kicked him out or not. But in the meantime, when he had lost his

00:28:42.900 college money and what he starts doing, because now he needs money to be able to play, is he became

00:28:48.220 like the errand person for the game. And so they'd be like, hey, Howard, will you go get me a sandwich

00:28:54.800 like at the deli downstairs or go get me some pizza or go get me cigarettes or go whatever. And he'd go

00:29:00.380 and do an errand and then he'd get a tip. And then he'd save his tips up and then get into the game.

00:29:04.800 Eventually, he didn't need to be the errand person anymore, because he'd turned the tip money into

00:29:09.140 enough money for him to eventually send himself to college for that brief period.

00:29:14.120 But so he moved from there to the Mayfair, which was

00:29:16.480 And the Mayfair is where?

00:29:17.460 Is it in Gramercy Park?

00:29:19.320 Okay.

00:29:19.620 Somewhere near there? It's either Gramercy or Chelsea. I think it's Gramercy Park. I think

00:29:24.120 it's near Gramercy Park. It's a nice place. Then he was playing there for a while. And he falls in at

00:29:28.500 that point with a group that really becomes very famous. Dan Harrington, main event champion.

00:29:35.580 Eric Seidel, who I think has nine world championship bracelets from the World Series of

00:29:40.640 Poker. It might be more than that now. But he's won, I think, like $40 million playing poker. I mean,

00:29:44.700 he's amazing. Steve Zolotow, also bracelet holder, was in that group. Jason Lester.

00:29:50.820 And these guys are all coming out of this one group. So this is sort of like Daniel Coyle

00:29:55.140 writes about these centers of excellence, or I forget what his term is for them. But these

00:29:58.940 little places that just produce so much greatness, like all these soccer players that come out of

00:30:04.020 this place in Brazil, and all these tennis players that come out of this place in Russia,

00:30:08.240 this became the place where everybody's sharpening everybody in this one place.

00:30:13.620 If you're a tourist and you hear about this game, you don't get to go and play, obviously,

00:30:17.080 right? Like they'll take your money if you want, but...

00:30:18.860 Yeah, anybody can play. And anybody does play. The thing about poker that's part of this stuff

00:30:24.480 about like the hidden information in luck is that that kind of uncertainty leaves open the

00:30:30.200 array of explanations for why you might not be doing well. So if I play chess against you a few

00:30:36.380 times, it becomes very clear to me why I'm losing. I'm losing because you're better than I am. I have

00:30:41.400 nothing else to protect my ego. Like I can't put a hat on it and say, no, it's because of something

00:30:46.400 else. See, that's such a great point that's worth pausing on it as well for a moment. There are very

00:30:51.800 few sports or activities that we play that are as pure as something like chess, where as you said,

00:30:59.620 there is zero excuse. Whereas I can't think of another activity. I'm sure there are several where

00:31:07.880 if you really want to, you can't, oh, the ref screwed me or, oh, the wind. I mean, did you see

00:31:12.960 that? Or like, I'm into archery, right? It's like, oh, did you see that animal move out of the way at the

00:31:16.880 last second? He heard the arrow coming or... Whereas in reality, no, you took a lousy shot.

00:31:21.240 Right. I mean, they're all on a spectrum. And one thing that we forget is that there's an

00:31:25.600 equanimity to luck. So if you're playing tennis and there's wind, your opponent is also playing

00:31:30.880 with that wind. If you're in a game, although we like to think so, the refs generally are making

00:31:36.340 bad calls, just sort of period, some good, some bad. And they're not like screwing a particular team,

00:31:43.440 systematically going against a particular team, although it sort of feels like that.

00:31:48.100 But what we notice is the wind blew our ball. An interesting thing that happens in sports that

00:31:53.220 are one-on-one, like I play tennis, is that you can blame it on the style matchup. That it's not

00:31:57.720 that you're not particularly good. That's a great new excuse. I hadn't even really thought of it.

00:32:00.880 No, you should use that. Like, oh, that was a bad style for me. I mean, I'm actually better than

00:32:04.760 them, but they're lobbing. They were doing a lot of lobbing and my high balls weren't really on that

00:32:09.980 day, but I would crush anybody else. So yeah, exactly.

00:32:12.900 So poker basically is just a fertile ground for that thinking.

00:32:17.620 Right. Which means that it doesn't cause you to kind of self-select out. As you said,

00:32:22.520 this was like a nest of what was going to become the best players in the world. And of course,

00:32:26.680 people were still coming back. So he ends up at the Mayfair.

00:32:29.980 Well, I guess there's another element here, which is the variable reinforcement of winning.

00:32:35.040 Wow.

00:32:35.480 So then you've got the whole dopamine side of this coupled with this psychology of I can post hoc

00:32:41.980 rationalize my way all day long. I can rationalize my losses, pump up my winnings from an ego

00:32:48.600 standpoint. And the next win is coming at a point I can't predict. I mean, that is a recipe for

00:32:54.460 get me into this.

00:32:56.280 Actually, let me just bite into that for a second. So this is actually really interesting. So let me just

00:33:00.940 say I didn't play poker at all in college. Like I never played. It wasn't until graduate school

00:33:05.820 that I played three times or something. So I go to graduate school at Penn. I'm studying cognitive

00:33:10.100 science. This becomes important to me later as I'm thinking about the game of poker, this issue that

00:33:15.440 you just brought up, the difference between a variable reinforcement schedule and a fixed

00:33:20.540 reinforcement schedule. What we can think about is we can talk about ratio or interval schedule.

00:33:25.740 So ratio schedule would be by number. So if you put a rat in a Skinner box, and for those who don't

00:33:33.160 know, Skinner box, it's got the lever. And then either there's positive reinforcement, like a food

00:33:37.560 pellet comes out, or maybe negative reinforcement, like a shock. So let's make it positive and say it's

00:33:42.200 a pellet. A ratio schedule is by number. So it would be like every fifth press, the rat gets a pellet.

00:33:49.980 And an interval schedule is by time. Every five minutes, the rat gets a pellet. And if it's fixed,

00:33:57.420 what it means is it's actually every five presses or every five minutes. And depending on whether it's

00:34:03.400 ratio or interval, you get different behaviors from the rat. So what happens on the ratio schedule

00:34:10.620 is that the rat has this very even pressing that goes on. This very even, somewhat quick, but not

00:34:17.920 mad. Press, press, press, press, food. Press, press, press, press, press. And then if it's every five

00:34:24.980 minutes, the rat twiddles its thumbs until it's almost five minutes, and then it goes crazy. So you get

00:34:31.760 this burst of activity. It's almost like they don't want to miss a second. They want to make sure that

00:34:37.620 they get that food exactly. They don't want to give up one second where they could have had food.

00:34:42.520 And so you get this big burst of activity. Then you can also take a ratio or interval schedule,

00:34:47.000 and instead of it being fixed, you can make it variable. And what that means is it's now on

00:34:51.240 average is the way that you can think about that. So let's say on average, every 20 lever presses,

00:34:56.520 you'll get food. Or on average, every five minutes, you'll get food. So now, instead of this big

00:35:04.420 burst of activity on the interval schedule, on average, every five minutes, because they're not

00:35:09.260 sort of waiting. And then, oh, I think it's about five minutes, and they go. Now they're pressing

00:35:14.160 pretty steadily the whole time, because they're not sure anymore, right? Is it going to be the second

00:35:18.860 or not? And then on the variable ratio schedule, you get the same thing. You get a lot of pressing

00:35:24.500 very quickly, because they don't know, is it the next press or not? So that's all kind of interesting.

00:35:28.840 And you'll see a different activity, depending on whether it's variable or not. What's interesting is

00:35:33.320 when you try to extinguish the behavior. This is what's really key. So when you extinguish a

00:35:38.620 behavior, what you do is you just withdraw the reward. So now you've trained the rat,

00:35:44.160 it's pressing the lever, and then you just stop giving the rat food. And the question,

00:35:48.140 what does it do? When it's fixed, it stops doing the thing super fast. It's like, hey,

00:35:54.260 wait a minute, like I've been pressing 100 presses of this stupid lever, you haven't given me any food,

00:36:00.360 I give up. I have figured this out. There's no food coming my way. Same thing if you're doing

00:36:06.140 it by time. But when you make it variable, they just don't stop. And it's particularly bad when

00:36:11.980 it's a variable ratio schedule. So I think about it like this. So anybody who's ever watched people

00:36:19.200 sitting at a bank of slot machines in Las Vegas, those are on a variable ratio schedule that you'll

00:36:24.780 get a reward, it's going to be on average, a certain number of plays, you could get a reward

00:36:30.420 five plays in a row, you could go 80 plays without really getting much in return. And you kind of don't

00:36:36.880 know, because it's on average. And what happens to people sitting at those slot machines is that

00:36:42.340 they'll be going along and there'll be no reward. And you'll hear them say a thing. I'm due.

00:36:48.320 I'm due. And what that means is that it doesn't matter what you do to that machine, you could just

00:36:55.860 turn it off. What they think is no, it must be the next press, it must be the next press. And this is

00:37:02.200 exactly what happens to the rats in the Skinner box, you train them that on average, every 20 presses,

00:37:08.380 they get food. And that means sometimes they're getting three pellets in a row. And sometimes they're

00:37:13.100 going 80 presses without getting food. That's obviously out at the tail, no pun intended,

00:37:18.460 but it's just an average. And now you would draw it and they just never stop pressing the lever.

00:37:24.260 And it looks just like what humans do. And I sort of always imagine the little rats in the cages saying

00:37:29.060 I'm due. So this is exactly the way that something like poker works. If we're thinking about sort of an

00:37:35.740 edge, what that means is that over time, I'm going to get a certain percentage of your money.

00:37:43.440 But in a particular moment, you could win a lot of hands in a row. In a particular moment,

00:37:49.660 you could not win so many hands in a row. And unless you're a really good statistical aggregator,

00:37:55.120 right, unless you're sort of stepping back and really looking at what does this look like over

00:37:59.880 the long run, it's really hard to see what the underlying distribution is. It's hard to see whether

00:38:04.300 you're winning or losing to that situation. Well, there's a slight difference, right? Because if I

00:38:07.940 understand correctly, the slot machine is driven by an underlying probability distribution curve.

00:38:14.240 So the person sitting there playing it like the Skinner rat is kind of right in that they are due,

00:38:21.620 they just don't know when. Whereas the person playing poker, there is no certainty that they

00:38:26.380 will ever win again necessarily. In other words, it's both luck and some skill. So they could take

00:38:34.120 probabilistic good hands and misplay them and lose on a potential hand that could have been their due,

00:38:40.840 right? Yes. I'm thinking about that. Yes.

00:38:43.060 Yeah. So it's sort of like in some ways, the person at the poker table has an even harder

00:38:47.860 problem than the person at the slot machine. Yes.

00:38:51.920 Because the slot machine is pure luck.

00:38:53.360 So the slot machine simplifies the problem because the machine is set to a particular distribution

00:39:00.380 and the probability is set about how much you're losing per dollar. And so if I sit down,

00:39:09.540 the underlying probability is knowable. That's right. And your only task is to pull the lever or

00:39:15.960 push the button. Whereas in poker, you still actually have to do what we're going to get to,

00:39:21.480 which is the skill of this game. Right. And what you're trying to figure out is,

00:39:26.240 I mean, obviously, this isn't true of everybody. But when people sit down at a slot machine,

00:39:30.340 people aren't thinking I won because I'm so skillful. I mean, occasionally they are if they

00:39:34.060 think they have a system. I don't think that's true. Most people are saying like, I'm doing this

00:39:37.920 because I'm going to get lucky and maybe I'm due and whatnot. So I think that you have less

00:39:42.120 illusion about it. So now in poker, you lay on top of the fact that it's stochastic,

00:39:47.820 that obviously it's a skill game. And in the long run, with enough iterations,

00:39:53.640 your results will be solely determined by skill, right? Because the probability starts to approach

00:39:59.800 equal distribution for everybody, assuming there's no cheating.

00:40:02.820 Well, it would be true even if there's cheating for this reason, that what that would mean is that

00:40:07.740 whatever skill elements that you were applying, you wouldn't be able to overcome

00:40:11.120 the situation. So all I mean by this is that you have an expected value,

00:40:15.460 that expected value is determined by your play compared to your opponent's play.

00:40:21.100 And when you say skill, do you mean two skills? The first being the ability to look at all the

00:40:29.520 cards that are available and calculate the probability of cards that are not being shown

00:40:35.600 and the skill of being able to bluff? Or are you referring more to the latter than the former?

00:40:41.740 Skill would be the umbrella of all the things that are within your control in the game that have an

00:40:47.980 effect on the outcome. So first of all, let me just step back a second. Sometimes people think that a

00:40:54.280 game isn't skill, if in the short run, there's a really strong influence of luck. So people will say,

00:41:01.700 well, baseball is a game of skill, but poker isn't a game of skill. Because they're looking at,

00:41:07.240 well, but in poker, I could have the best hand, I could play it perfectly, and I can lose to you

00:41:14.540 because of the turn of a card. So therefore, isn't it just luck? And how can you tell if something is

00:41:20.840 a game of skill? Because I'm obviously making the assertion that in the long run, your results are

00:41:25.320 going to be solely determined by skill.

00:41:27.240 I think the way you said it actually is probably the best definition I've ever heard, which is

00:41:30.200 in the long run, if two different people can play the exact same game over and over and over again,

00:41:35.540 and have dramatically different results under the same stochastic input. Doesn't that imply that

00:41:42.280 there is a difference that is in the control of the player? And isn't that the definition of skill?

00:41:47.480 Yes. Because of actions that they take, their distribution is different.

00:41:51.100 Right. So for example, if you and I made up a new game, which would be a really boring game called

00:41:54.680 Toss Two Dice, and the person who gets the higher number wins. If you and I played this game for the

00:42:01.760 next week, we sat here at my table, and we played over and over and over again, in the end, we would

00:42:07.420 end up the same.

00:42:07.820 There'd be nothing that we could do that would change the outcome.

00:42:09.840 And therefore, there is no skill in that game.

00:42:11.880 There's no skill in that game. But if we can bet on it.

00:42:14.160 That's right. And baseball, of course, is such an obvious example of skill. Because while there is

00:42:18.940 luck involved, within a matter of minutes, you would tell the difference between me and a

00:42:23.740 professional baseball player doing anything on a baseball field.

00:42:26.660 There's a really great test that you can do to figure out if something is skill. And then let

00:42:31.960 me just get back to baseball for a second to show what's happening, why people are sort of

00:42:36.540 misperceiving with poker, whether it's a skill game or a luck game. So the first thing is you just

00:42:41.540 want to apply a test. Can you lose on purpose? Assuming you followed the rules of the game. Now,

00:42:46.100 it's not symmetric. I cannot apply a test of can I win on purpose, but I can apply a test of can I

00:42:51.880 lose on purpose. And I'll tell you why. So tic-tac-toe, let's agree that that's a skill

00:42:55.720 game. It's a bottom feeding skill game, but it is. Right. But if you and I play, neither of us can

00:43:01.820 win on purpose. Assuming that we both know how to play the game, because we're going to tie every

00:43:06.220 single time. But one of us can lose on purpose. Very easy to lose on purpose at that game.

00:43:10.580 So we don't want to apply the win on purpose, because that's hard, particularly as you narrow the

00:43:14.340 skill gap, then you can't do that. But I can lose on purpose. And in a game like baseball,

00:43:19.040 you can lose on purpose. In a game like basketball, in a game like chess, in any of these

00:43:24.820 games, you can lose on purpose. We can think of other acts of skill. Like if you're a salesperson,

00:43:30.260 you could make it so that on purpose, you never closed a sale. As an example, it's a game of skill.

00:43:36.060 If you were a lawyer, you could probably make sure you lost every case. I mean, hopefully you don't do

00:43:40.580 this. And in poker, you can indeed. I can lose on purpose very easily. I can't win on purpose,

00:43:45.980 but I can lose on purpose. And that's true of baseball as well. Somebody can lose on purpose,

00:43:50.300 but they cannot win on purpose. That you do not control. That at a basic level is a really good

00:43:56.900 test of whether something is a skill game or something is a luck game. Where I think the

00:44:01.020 confusion comes in is that as you narrow the skill gap, luck starts to play a much bigger role. It

00:44:08.440 starts to kind of show itself that influence of, hey, there's different ways that this could turn out.

00:44:14.700 And sometimes you're going to see one thing and sometimes you're going to see another thing and

00:44:18.880 you kind of don't have control over which thing you happen to see a lot of the time.

00:44:22.580 We start to see that influence as we narrow the skill gap. So that's a great point, right? That's why

00:44:27.820 you look at virtually every Superbowl these days. Most of them come down to the last series,

00:44:33.820 the last possession. I don't remember these stats, but it could be as high as like two thirds of Superbowls

00:44:39.100 come down to the last possession. And if you think about that, you could easily do what I do

00:44:44.660 when looking at the Eagles beating the Patriots and say it came down to that one stupid play when

00:44:50.100 Brady went back for the pass and boom, the ball gets stripped out of his hand. And it's like,

00:44:53.740 if that didn't happen, he absolutely would have connected with Gronk. They absolutely would have

00:44:58.140 won. And whether or not I'm being delusional or not, you can't deny the point that those teams were

00:45:02.980 so damn near perfect. If you took the Red Sox and you had them play a Little League team,

00:45:09.940 there'd be basically no influence of luck, right? Like what luck is going to go the way

00:45:14.160 of the Little League team? I mean, with the Red Sox, what luck is going to go against them?

00:45:19.980 Aside from all their players all at once get injured and can't play anymore, they could put in

00:45:26.340 their third stringers and the Little League team is crushed. But if you put the Red Sox,

00:45:32.660 against the Yankees, this is why we have to have seven games. And it's probably still not enough

00:45:40.400 to try to figure out who's supposed to advance to the next round. And in fact, we know it's very

00:45:45.860 rare in a series that a team wins for nothing. It's very often going to the sixth or seventh game.

00:45:52.800 Why? Because it's very narrow. And it's very often things like you just said, who wins the coin toss

00:45:58.000 in a football game? That has a huge effect on what the outcome is. Who just gets to go first?

00:46:02.400 Oh, especially in overtime.

00:46:03.520 Overtime. Yeah.

00:46:04.260 Oh my gosh. That's a huge influence of luck. Why? Because you've narrowed the skill gap so much.

00:46:08.900 But if you took that NFL football team and you put them against peewee players.

00:46:14.280 Or even a college team.

00:46:15.380 Or even a college team.

00:46:16.300 It's unbelievable that gap.

00:46:17.380 The coin toss would have no effect. No one would care who won the coin toss in overtime. The NFL team,

00:46:24.280 I mean, I don't know how they would get to overtime, but you just magically make overtime appear.

00:46:28.720 You put them in there and the coin toss will have no effect whatsoever.

00:46:32.980 So I think this is something for people to really understand. And so if we think about a game like

00:46:37.240 poker and a lot of these decision-making processes that we do in life, the skill gap is actually quite

00:46:41.880 narrow. So once I've taught you in poker things like, you know, the ranking of the hands, like a

00:46:48.520 flush beats a straight at you. You know, you can bluff, you know, sort of about betting. You can read your

00:46:54.180 hand. All these things. That's so much of the building blocks of the game in terms of the skill.

00:46:59.060 And how long does it take? Let's hit pause for one second. And just for the person listening to

00:47:03.300 this who maybe doesn't know the ins and outs of poker. When we're talking about poker, you're

00:47:07.960 talking about a variant of poker called Texas Hold'em, correct?

00:47:11.080 Not necessarily, but we can.

00:47:12.580 Okay. That's what you played, correct? And that's what most of the pros are playing when we turn TV on

00:47:16.680 and stuff like that?

00:47:17.280 So what you see most of the pros playing when you turn the television on is Texas Hold'em. In terms of what

00:47:22.320 most of them are playing, it's not necessarily that game. That's one of the games that they play.

00:47:27.080 So give us the brief overview of how that game works so that we can understand what those building

00:47:31.580 blocks are that you're talking about, which is everybody within a day could know in detail what

00:47:36.720 you're explaining quickly about how this game works.

00:47:38.880 So this isn't true of all poker games, but it's true of almost all poker games, enough that we can

00:47:45.180 pretend that it's kind of true across the board, that the goal is to make a five card hand. Again,

00:47:50.160 there are some exceptions to this. I just want to make that clear, but I'm capturing most of poker.

00:47:54.180 If I say you want to make a five card hand, those five card hands range from none of the five cards

00:47:59.960 match up. And so all you care about is the top card and the hand. In most poker games, aces are high.

00:48:09.300 In some poker games, they're low, but mostly they're high. Sometimes they're high and low. It depends.

00:48:14.660 So you're trying to make a five card hand. Those five card hands range from very bad, meaning none

00:48:21.120 of the cards match up to each other in any way, to really, really good. And really, really good

00:48:26.980 would be a straight, which would be five cards in rank order. Five, six, seven, eight, nine. That

00:48:34.260 would be a straight. Ten, jack, queen, king, ace. That would be a straight. You can have a flush.

00:48:39.220 That's when all five of the cards are of the same suit. So you have five hearts or five spades or

00:48:43.700 whatever. Full houses where you have three of one rank, king, king, king, and then two of another

00:48:49.600 rank, 10, 10. So your five cards would be king, king, king, 10, 10. So anyway, people can go and

00:48:55.960 look up the hand rankings. The highest rank is a straight flush? It's a royal straight flush. So

00:49:00.280 that would be an ace, king, queen, jack, 10, all of the same suit. All the same suit. Probability of that hand?

00:49:06.720 I'm not sure what it is, but a straight flush is one in 6,000 something. It hardly ever happens.

00:49:13.360 So let's start there. So basically, almost any poker game you play, you're trying to make a five

00:49:17.840 card hand. And there's a very clear set of rules about what the hierarchy and ranking are. There

00:49:22.860 is no ambiguity. There's no ambiguity when you're playing. Now, different games have different

00:49:27.320 rankings because you can play low. So in low, you're actually trying to make sort of what in high

00:49:31.480 would be a bad hand. Okay. So then the other thing about most poker games is for most poker

00:49:37.060 games, you're trying to make a five card hand out of either seven or nine possible cards.

00:49:42.880 That just is also true of most poker games, not all. So in Texas Hold'em, it's a particular form

00:49:50.060 of poker where you have seven cards to work with to try to make your best five. So you're sort of like

00:49:55.720 tossing two of them. But some of the cards, everybody gets to use. They're called community

00:50:03.340 cards. So we can think about the variance of poker. So when you watch something like the Cincinnati kid,

00:50:11.120 for example, Steve McQueen, in that particular game, people are dealt their own five cards. And so

00:50:18.040 you don't get to use any of the five cards that I have. So our cards are unique from each other,

00:50:23.220 and you don't get to see any of my cards. Then you can have variants, which are called stud games,

00:50:29.180 where all of our cards are private, but you get to see some of my cards. So in a stud game,

00:50:35.680 at the end of the hand, I'll have three cards that are face down and four cards that are face up to you.

00:50:42.880 And I'm still making my best five of those seven. And I don't get to use yours. It just gives me more

00:50:48.380 information. It just gives you more information. So once you can see the four cards that are up,

00:50:53.220 gives you some definition to the kinds of hands that I can have. So as an example, because I only

00:50:58.060 have three down, if my four up cards are a spade, a club, a heart, and a diamond, I can't possibly

00:51:06.680 have a flush because the most I could have of any suit now is only four. So you get certain information

00:51:12.780 about the possibilities for what I could be holding because you get to see some of my cards. But yet,

00:51:17.920 you still don't get to use any of my cards. Whereas in Texas Hold'em...

00:51:21.160 Now that's a stud game.

00:51:22.760 But it's common.

00:51:23.760 But it's common. So what happens is you get dealt two cards that are private to you. So these are

00:51:28.140 face down. You don't get to see those particular cards. And then in a particular way, five cards

00:51:34.980 eventually will end up in the middle. So notice that that's seven. But those five cards that are

00:51:39.620 sitting in the middle, both of us get to use. Now, what that means is that these five cards in the

00:51:45.700 middle define for me what your possible holdings are. So that now changes for me what I think that

00:51:53.880 the strength of my hand is. I'll give you a simple example. It just turns out if you have two private

00:51:59.200 cards, the most of a suit that I could have in my hand would be two. So I could have like two hearts,

00:52:04.940 which means that in order for me to ever make a flush, which is five of the same suit, like five

00:52:10.120 hearts, I need to have three more of those out on the board. So if I look on the board and

00:52:15.560 you look on the board, and we both see that there aren't three of a suit that's on the board,

00:52:19.580 it eliminates the possibility of a flush from the other person's hand. That becomes important

00:52:25.000 because if I have a hand that is vulnerable to a flush, like if I have three of a kind or two pair

00:52:32.100 or something like that, that now increases how I think about the strength of my own hand.

00:52:39.820 And you get to look at your cards as soon as they're dealt the two as soon as they're dealt,

00:52:42.920 you get to see your private cards, right? So you know what your private

00:52:45.500 cards are the whole time. And then these community cards are coming down in the middle

00:52:50.020 and those community cards are defining for you what you think the other player might have.

00:52:55.700 Now, this is a lot of where the skill in poker comes from is understanding what those community

00:53:01.600 cards mean. Understanding how they define what the other player has, how they might relate in a

00:53:08.940 logical way to what the other player has, what the probability is that the other player's cards match

00:53:14.160 up in some way to those five cards in the middle. Yeah, it's both, right? It's what do those cards

00:53:19.220 mean about what you have? And what do they not mean about what other players have? Right. And

00:53:24.700 it's interesting because depending on how many players are at the table, and what's the typical

00:53:29.920 number of players that are at the table? And hold them? Yeah. It's generally nine, sometimes it's

00:53:34.680 eight. The higher the limit, they tend to limit the number of players more. So if you're playing in a much

00:53:40.340 lower limit game, you might see 10 people at a table. If you're playing in a much higher limit

00:53:46.000 game, it's more likely to be eight. Which is still kind of staggering to me because now, so you have

00:53:52.420 this math of here's what I have, here's what's common. Based on what's common, what can I impute

00:53:58.800 about everybody else? But now I actually have to pay attention to the behavior of the other people.

00:54:03.560 So I can do the probabilistic calculation broadly. I'm doing two probabilistic calculations. The

00:54:10.680 implications of these cards on me, the implications of these cards on people for whom I don't know the

00:54:15.600 other two cards. But now I have to do eight different behavioral checks. Right. So one thing

00:54:21.340 about the way they relate to your own hand is probabilistic and one isn't. So the thing that

00:54:26.040 isn't probabilistic is these relate to my cards, my hand in a certain way. So if I'm looking at the

00:54:31.160 middle hand, like I know what I have, hopefully. But what is probabilistic, and this is what's

00:54:37.400 really important, is what does that mean for whether I have the best hand or not? So that I

00:54:42.120 don't know, because that's relative to your hand, because there isn't an absolute ranking. It's not

00:54:46.740 like, oh, I have a flush, I get to win automatically. Now, that comes up, like sometimes if I look at the

00:54:54.080 center of the board, I have the very best possible hand that a person could hold. That's very rare that

00:54:59.960 that occurs. Most of the time, I'm somewhere in between the worst possible hand that a person could

00:55:05.900 hold, and the best possible hand that a person can hold. So I have to figure out where I'm sitting on

00:55:10.900 that spectrum, right? And it's probabilistic in nature, whether I'm likely to be beating you or

00:55:15.660 not. And I can think about that in the sense of sort of just base rates, like if I'm holding this

00:55:21.240 strength hand, how often would I be winning? But then I also have to adjust those probabilities for

00:55:27.160 what you're doing and how you're acting. And how you're acting is going to tell me something,

00:55:32.900 two people could act in the exact same way. But depending on my model of who they are,

00:55:37.980 it could mean very different things. So I could bet strongly with a hand. And if you call,

00:55:43.100 that could mean that you have a particular range of hands, that's x. And if another person calls their

00:55:51.420 range of hands that they might call with could be narrower or wider, they could be willing to call with

00:55:56.020 either a hand that's much worse than the bottom end of the range of hands that you'd be willing to

00:55:59.900 call there with. Or they could actually be calling with where their bottom range is better than what

00:56:06.640 your bottom range is. And that's going to depend on that person.

00:56:09.200 As a general rule, does a newcomer to a game have an advantage or a disadvantage? Let's say you and

00:56:14.060 six of your friends have a regular game going, and I show up as a first timer. You guys have never

00:56:20.000 played with me. Does that give me an advantage or disadvantage in terms of the point you just made about

00:56:24.940 ranging me? All things being equal. So let's pretend you know nothing about this. Until we

00:56:29.800 play enough, you don't know if I'm good or not. So let's assume I know the fundamentals of the game.

00:56:34.300 I would say all things being equal against the game, you're at a disadvantage because you don't

00:56:39.920 know how to range six people. But me personally, I would prefer to play in terms of my ability to

00:56:46.460 range you. I'm going to be better with somebody I've already played with before. It depends on what

00:56:50.240 question you're asking. And in fact, there's actually been many, many cases in poker where

00:56:56.240 someone has come in and they're doing something that's just really unusual, that is not something

00:57:05.280 that would be expected. And they do really well a certain period of time. And then the market

00:57:11.600 basically kind of figures out what they're doing. Either they adjust or they don't. But I've seen a lot

00:57:17.000 of players come in and win a lot. In some cases, it's just attributable to their short-term bursts

00:57:22.780 of luck. But in some cases, they're actually doing something really unexpected. And people are

00:57:28.360 actually mis-ranging them. This usually is because they're playing more aggressively with hands that

00:57:34.080 people normally wouldn't play that aggressively with. And then people sort of figure it out and

00:57:38.560 they adjust. And very often that person starts losing after that. Sometimes they adjust and they do

00:57:43.540 other things. But often it has to do with this problem of sort of what do you have?

00:57:48.020 For example, like in cycling, a totally unknown cyclist can win a stage of a race because they

00:57:53.780 lead out, they break out. And the Peloton sort of says, well, there's no way this person can do such

00:57:59.260 and such because we don't know anything about them. And all of a sudden they let them get away.

00:58:03.380 Now they're not going to be able to do that the next day because all of a sudden the Peloton is going

00:58:06.520 to say, well, wait a minute, wait a minute. Now we're going to treat you like we would treat anybody

00:58:09.660 who has the potential to win and your moves now fit into a different frame.

00:58:14.300 Right. Exactly. That's exactly what happens in poker. The first thing that you're trying to do

00:58:18.640 is figure out, I know sort of in an absolute sense what I have, but I don't know where that sits

00:58:24.320 relative to what you have. And then once I sort of figure that out and that's probabilistic in nature,

00:58:29.780 I now have to figure out for you what's the best line of play for me.

00:58:34.680 And tell me how the flow of the game goes from here. So the five cards common,

00:58:38.200 we each have our two. How does the betting work and what changes with the showing of cards or

00:58:44.060 movement of common cards? So you start off, you get dealt two cards and there's a round of betting

00:58:49.660 and you can choose at that point. And this is actually a really important concept in poker.

00:58:54.400 I actually tweeted about a little while ago because I think people should be talking about it more,

00:58:58.200 which is you're totally allowed to fold. Remember when Steve Dubner and Stephen Levitt wrote about

00:59:02.860 this in Freakonomics, like the benefit of quitting. We do not spend enough time encouraging people to

00:59:07.260 quit. No, we spend a lot of time encouraging people to stick to it. And it's actually for

00:59:12.620 everything that you stick to, there's huge opportunity costs involved, right? There's all

00:59:16.580 the other things that you could have done. And if you're sticking to something that you're negative

00:59:20.700 expectancy to, and I'm not saying money, I mean, I'm saying you might be negative expected happiness

00:59:25.160 to it or negative expected health to it or negative expected whatever that, and you're giving up the

00:59:30.820 opportunity to go do something. I mean, I'm a big fan of quit fast. So what I like, my theory is do lots

00:59:37.740 and lots of things that are basically a free roll where there's very little downside to it. Go try out

00:59:42.340 piano, right? It's like a little bit of your time. You can't die doing it. Just go see if you like it,

00:59:48.740 but then quit fast. Just quit fast and move on to something else. And then if you find out,

00:59:54.100 oh, I really love this. I really want to spend my time doing it. Now stick to it. So I'm a huge fan of

00:59:59.100 that. But in poker, this idea is very natural for a poker player because you're doing a lot

01:00:03.320 of the quitting part. You're just saying like, no, this isn't a hand that I mathematically want to

01:00:07.040 play. I don't think I realized that. And that actually might answer this question, which is

01:00:11.680 one of the things I've never understood about gambling in general is you lose much more than

01:00:16.080 you win, presumably. No, not in poker. Ah, because of what you just said. Because you can cut your

01:00:21.420 losses. Okay. That answers the question because my question was going to be, I think it answers my

01:00:24.800 question. Given the power of loss aversion, how the hell do people keep playing? Because you'd think

01:00:32.180 if you only win once out of every 10 hands, that's nine experiences that are each more painful than

01:00:40.100 the pleasure of the win. But of course you don't view those nine as losses. You just view them as

01:00:44.720 non wins. You do lose real hands. I mean, you get to the end of a hand and you lose real money.

01:00:50.160 But a lot of the times, if you really feel like, you know, the table, well, you know,

01:00:53.300 the dynamic, well, you just cut your losses and you're like, I'm going to let these two guys fight

01:00:56.640 it out over here. But right. Exactly. And I'm going to only go to the mat when I really have a shot to

01:01:00.700 win. And presumably losing that still falls into the loss aversion pain. By that point, the pretest

01:01:06.940 probability is such that you have a better shot. So it comes back to this kind of variable ratio

01:01:12.700 reward schedule. You're winning enough big pots. If you're playing this particular form of poker,

01:01:18.040 which is limit, let me just say something about poker comes in two styles. One is limited

01:01:23.120 betting. So in limited betting, it means that, for example, if we're playing, maybe we have to bet

01:01:28.340 in like $10 increment. So I can bet $10. And then if you want to raise the bet, you can make it $20.

01:01:34.040 And then if I want to raise it, I could raise it to 30. In other words, we're raising in $10 increments,

01:01:38.280 and we can't do anything more than that. What comes along with that is there's limited exposure

01:01:42.540 to the person, but it obviously limits what I can win from you on a particular hand. And then there's

01:01:47.960 no limit, which is what you really see on television, which is the super exciting. Someone shoves a

01:01:53.100 million in chips on.

01:01:54.240 Like when I think of poker, I think of rounders. I'm sure there's been a million movies.

01:01:57.580 Which is no limit. No limit and snapping Oreos.

01:02:00.820 I want to be around John Malkovich. I want Oreos. And I want to hear a Russian accent. Like that's

01:02:06.680 all right. Anytime that movie's on anywhere, like if I turn on Netflix, and somehow it pops up,

01:02:11.180 I'm watching it.

01:02:12.340 Right? Yeah, that's no limit, which is like that super sexy.

01:02:15.380 Oh my God, it's incredible.

01:02:17.160 I'm going to lay everything on the table. Can I bet my lung? But there's also something called

01:02:23.280 limit. So in limit, a really good limit player at the end of an eight hour session, a very good limit

01:02:31.680 player is going to have won about 56% of the time. This is a really, really excellent player. Now go talk

01:02:38.120 to anybody on Wall Street. They can get a hundred billion AUM if they say for every bet has a

01:02:45.360 56% chance of winning, right? I mean, this is a huge ad. So I'm saying this is a really,

01:02:49.740 really, really good limit player. What that means though, is that the person that they're

01:02:53.400 playing against is going to win 44% of the time. So think about how much reinforcement they're

01:02:58.180 getting. So sometimes they're going to win three sessions in a row, right? I mean, 44%,

01:03:01.960 that's a lot. This is a great player.

01:03:04.740 That's hard to believe, even when you bring it back to the context of what we keep using as a

01:03:08.240 great comparison, which is sports. Think about this. If the New England Patriots played,

01:03:12.740 let's see, who was the worst team in the league last year? I don't even remember. If you put the

01:03:16.680 best team against the worst team, it would not be 56, 44. It wouldn't even be close. It would be

01:03:21.860 85, 15 or 90, 10, even in professional sports. So that tells you how much narrower the gap is here

01:03:29.780 in poker. This is specifically in this limit version of poker. Now in no limit, you can get these bigger

01:03:35.220 spreads. Ah, the 56% is dollars, not number of times in which a person wins.

01:03:42.020 56% of the time is at the end of an eight hour session. How many times have I won?

01:03:46.540 Have you won? Okay.

01:03:47.500 At the end of that session. So over eight hours. Now, just to allow you to understand,

01:03:52.920 it's just a matter of the number of iterations because it's sort of in the same way that if

01:03:57.400 the Patriots are 56% against another team, if they play enough times, they're 100% to have won the

01:04:04.400 series. So a poker player who's going to win 56% of the time at the end of eight hours, at the end of

01:04:09.560 1500 hours will approach to 100% that they'll be in the money at the end of the year. And this is

01:04:14.520 in this limit version of poker. Now, when we get to no limit, basically what's happening is that

01:04:19.120 you're realizing, you're more likely to realize you're... You're going to see huge volatility and

01:04:24.540 you have enormously asymmetric outcomes at times. Right. So here's an interesting thing,

01:04:28.860 actually, because of that, is that when I started playing poker... I love that we haven't even got to

01:04:33.640 you playing poker yet, but that's okay because we're going to get there. When I started playing

01:04:36.760 poker, the pool of players was small. So it wasn't on TV. It wasn't on the internet. There just wasn't

01:04:44.460 this huge market of people who were like, oh, I'm going to go and play poker. So what that meant was

01:04:51.240 that this limit version of poker was really what was being played in casinos by professionals.

01:05:02.160 And the reason was that when you had someone come in who was not such a great player, making sure

01:05:09.080 they were getting that frequent reinforcement was really, really, really important. So you can think

01:05:14.980 about it this way. If you have kind of an unlimited supply of new players coming into the game, when you

01:05:20.360 come in, I should just take all your money right then because who cares if you get depressed and

01:05:24.760 go away? Who cares if I hit you over the head and it plays more like chess and you're like, well,

01:05:32.020 I'm not going to play anymore because... It's not even close. It's not even close. That's ridiculous

01:05:35.860 because there's literally 17 people standing behind you waiting to take your seat. So now what I can do

01:05:41.940 is I can just maximize in that moment. I can just take as much of your money as quickly as possible

01:05:46.120 because there's going to be somebody else willing to come into the game. When I first started

01:05:49.660 playing, that was not true. When I first started playing, if someone came in the game who was

01:05:53.460 really playing at negative expected value, you didn't want to take a sledgehammer to them.

01:05:59.000 You wanted to make sure that they were enjoying themselves, by the way. And so playing these

01:06:04.440 limit games, which were kind of like a softer way that the edge played out over a longer period of

01:06:09.800 time, was kind of the way to go because then they won enough. It was like they won enough of the

01:06:15.740 time to feel pretty good. And the thing is that I think that people focus when they're thinking about

01:06:21.380 poker on the expected value being about money. And when this person came in and they were losing

01:06:28.560 money, it wasn't that awful for them. When somebody goes to see a sports game, they play money. They get

01:06:33.760 something in return for it, right? They get to watch a good game. When somebody goes to a restaurant

01:06:39.120 and they order food, they're giving money.

01:06:43.380 Yeah, there's a guarantee expected loss.

01:06:45.180 Right. But for them, what they're getting in return is of value. So generally, when these people

01:06:50.860 would come into the game who would be playing at negative expected value, if you only thought about

01:06:54.680 money, I think that most of them were actually playing at positive expectancy when you looked

01:06:59.400 overall at what their overall experience was. Now, the thing is that no matter who you are, it's not fun

01:07:04.440 for somebody to hit you over the head with a sledgehammer. Like here, just give me all your money.

01:07:08.260 It happened in two seconds. Now go away. But when they would come in and they'd get to play with

01:07:14.460 really good players and they'd be learning the game, which is exciting. And they'd feel like,

01:07:19.260 you know, I'm playing at table one at the Bellagio or Aria or whatever. And they're winning enough

01:07:25.420 to feel good. They're probably learning. And this is entertainment for them. Now it's no longer zero sum

01:07:33.420 anymore. It's like, is it zero sum as far as the money is concerned? Sure. But it's not

01:07:38.240 zero sum overall when you look at like, what's the scope of what people value, because there's all

01:07:42.540 this entertainment happening as well. And I remember there was this guy, I won't say his last name,

01:07:46.780 but this guy named Jay, who came and played, he came in the late 90s. And he, you know, he's probably

01:07:52.320 mid to late 70s. He had made a bunch of money on Wall Street, moved to Las Vegas to retire. His health was

01:08:01.440 okay. His mind was super sharp. And I think that he just sort of decided this is what he wanted to do

01:08:08.300 for his retirement. He had a ton of money. And so he started playing in these high limit games with

01:08:13.360 the high limit players at that time. And it was a small group. And he'd come in and play. And the

01:08:19.840 whole group would break for dinner. And we'd all go with Jay to go eat at a restaurant in the Mirage.

01:08:26.740 It was at this time. This was before the Bellagia was built. We'd all go eat dinner. And he would

01:08:31.240 sometimes have parties at his house. And we'd go over to the party at his house, or he'd get invited

01:08:35.380 to somebody else's house. And this was his social life. And he was losing a lot of money, but he was

01:08:40.800 happy. He was having fun. He had this whole group of people. And he got to have this fun conversation.

01:08:46.960 And he got to learn this game and so on and so forth. And I think this is kind of how he spent the last

01:08:51.740 four or five years of his life. And he was for sure winning to that. So I feel like people sort

01:08:57.420 of think it all sounds very cutthroat. But it's not. And this is something that certainly I think

01:09:02.780 that the players in the 90s, for sure, were really thinking about, which is what's the value add? How

01:09:08.380 are you bringing value for this? We know how they're bringing value to us.

01:09:12.420 Wow, that's so interesting. I wouldn't have guessed that. But that's a nice symbiosis.

01:09:15.440 There's a really, really famous player named Chip Reese. He sadly passed away about,

01:09:20.880 very unexpectedly, he was young. Passed away probably about 15 years ago now. Maybe the

01:09:25.760 greatest poker mind ever. I mean, certainly one of them. And the thing about Chip was he just made

01:09:31.620 everybody happy around him. He made a ton of money playing backgammon. He made lots of money playing

01:09:37.280 poker. And he had the superpower of these people who were very negative expectancy, if you were thinking

01:09:42.820 about money, would be calling him up like dying to play with him. Because he was nice. It was just

01:09:48.020 a really good time. I think about it as the same thing. Why do you go to a movie? Well, okay, if we

01:09:53.400 just think about it as money, it seems like a losing proposition. But of course, that's not what we should

01:09:58.520 be really thinking about that as. And so I think that that gets lost a little bit as the game grows and

01:10:04.020 becomes really, really, really big. And so if you don't have a particularly good time, and you're sad,

01:10:11.140 and you leave, there's somebody else just to take your place. And I think that that interesting

01:10:17.460 aspect of how it really wasn't very zero sum at all kind of goes away. I think it's come back a

01:10:23.960 little bit now because the poker economy has contracted. And I think they're sort of coming

01:10:28.680 back to that idea of how does this become fun for everybody? It's a really interesting aspect of the

01:10:33.840 game back then. So when did you figure out you had a knack for this? It's hard to say because I feel

01:10:38.200 like it's all relative. You know, I retired in 2012. And pretty much anything else that I can

01:10:44.320 think of that's competitive, poker's really changed the ability for people to be able to

01:10:50.600 run batches of hands, for example, after the fact is sort of think, oh, here is the situation that I

01:10:56.200 was in. Now I'm going to go and I'm going to run this and sort of see what the right answer would

01:11:00.300 have been so that I can take that into the next time I play. The kind of analytics that people have,

01:11:05.040 the ability for them to discover what the equilibrium point is on a particular decision

01:11:10.360 or whatever. It's just beyond what I can even comprehend. So I left before I would have had to

01:11:16.880 put that work in. If I did put the work in under those conditions, I'm not sure that I would

01:11:22.260 do well. I mean, I know that if I sat down today without doing that work, I would get completely

01:11:26.800 crushed. So in that sense, you can look at me and say, well, I didn't have a knack for it. If I tried

01:11:31.920 to play against today's players who know the things that they know and are able to run the

01:11:36.960 kinds of simulations and analytics that they can run now and they're understanding the mathematics

01:11:42.880 of the game are and sort of what the optionality is. So you're saying that a professional player

01:11:47.500 today will take every hand of a game and they'll have the simulation after the fact of that hand?

01:11:53.240 Well, not necessarily every hand, but they can pick hands and they can actually run the same way

01:11:57.440 that you do it in baseball or... Well, I was going to say, this sounds a lot like Moneyball,

01:12:00.780 which is the advent of analytics to the game. Right, exactly. So they can sit there and they

01:12:05.660 can say, they can just set the parameters. If I am in a particular situation where I think that I have

01:12:11.500 to bluff and if I give you this range of hands, how often do you fold? If I give you this range of

01:12:16.660 hands, how often do you fold? And they can figure out exactly at what point, if I bet a particular

01:12:23.160 amount, am I winning enough to that bet that I can now try to bluff in that situation that maybe

01:12:28.740 I would have never thought in a million years was a situation that I would be winning to if I were

01:12:32.740 to bluff there because I just, I didn't have the data. So I feel like knack is such a strange word

01:12:39.300 because I feel like it's knack relative to what? Well, to your peers at the time. Yeah. So those peers

01:12:45.760 changed. So I started playing in Montana. What year is this? Oh my gosh. I started playing,

01:12:50.600 I turned professional in 94. So it's right around that time. And I was playing against people in

01:12:55.740 Montana who a lot of them were retirees who were, this is kind of what they did. Some people have

01:13:01.920 their bridge game or some people are playing golf or some people are playing poker and that's kind of

01:13:07.000 what they wanted to do. So while I think that pretty much everybody who plays poker would like to get

01:13:11.680 better and enjoys the game more, some people are more hungry to do that than others. And people are

01:13:18.920 getting different things out of the game and the people that I was playing with that were not

01:13:23.960 thinking, let me become a world-class player. At that time, particularly because I had really

01:13:30.200 great mentors, my understanding of the mathematics of that game was so beyond their understanding. I

01:13:36.260 don't think that was part of their enjoyment. So the day that I stepped into that room, I had a knack

01:13:42.500 certainly compared to the people that I was playing against. I started playing two years before that in

01:13:47.580 that game and that was sort of where I was learning. And then in 94, I decided I was a professional

01:13:52.740 poker player and I moved to Las Vegas and I started playing there. And this is how many years after the

01:13:57.040 game in New York that, or was it only your brother that was playing? Only my brother was playing. Yeah,

01:14:01.240 I never played in New York. I don't think I ever played a hand of poker in New York. So you get to

01:14:05.440 Vegas in 96? No, I got to Vegas in 94. Oh, that was the declaration of running pro. Okay, got it.

01:14:11.100 Does it become less fun when it's your job now? Does it amplify the stakes? Eventually, but no,

01:14:16.340 not at this time. Poker is a really complicated game. Very, very complex. And I know that there's

01:14:24.140 just so much that I don't know about the game. And I'm totally aware that if I tried to sit down and

01:14:29.320 play against today's players, that they would bury me because there's this whole layer of the game that

01:14:35.000 I haven't discovered. But while I was particularly in those early years in the 90s, I had started off

01:14:42.840 in Montana playing only limit Hold'em. Now I go and I start to learn more about that game. So I'm

01:14:49.200 starting to sort of peel back the layers of that game. And like most things that you do, as you peel

01:14:54.400 those layers back, what you start to see is that there was a whole bunch of stuff underneath it that

01:14:58.020 you didn't even know existed. Right? So you think that the onion is a certain size, and you peel a

01:15:03.340 layer back and you realize, oh, wait, no, there's like so many more layers under there than I ever could

01:15:07.380 have thought. I was telling this to somebody the other day, my head of research always says,

01:15:11.120 I don't know if he's paraphrasing somebody or if he coined the term, but the further you get from

01:15:15.100 shore, the deeper the water. Somebody asked me a question about a topic. I don't remember what it

01:15:19.060 was. But I said, and I wasn't being facetious, I really mean it. I know less about this subject

01:15:24.720 today than I did 10 years ago. And they said, how is that possible? And I said, well, to be completely

01:15:29.560 accurate, I mean, on a relative basis, obviously, on an absolute basis, I know more than I did 10 years

01:15:34.160 ago about subject why. But my knowledge of how much broader it is today has dwarfed my absolute

01:15:42.220 knowledge. So on a relative basis, I've gone backwards. Yeah, exactly. That's what's happening.

01:15:47.760 So I'm learning this game limit hold them, learning things about poker in general. And then I start

01:15:53.260 expanding my repertoire of the types of games that I play. So I start to learn the Omaha games,

01:15:58.120 I start to learn the stud games, I start to learn pot limit and no limit. So those are different

01:16:03.040 betting structures. And I'm immersed. It's like I'm living and breathing and talking poker

01:16:08.720 all the time with peers, with mentors, so on and so forth. So it's not really like a job in that

01:16:17.640 sense. Certainly not during that first eight or 10 years that I'm playing, I'm learning. And every

01:16:25.760 second is learning. So when did you see sort of robots in the book, but you also figure out

01:16:30.900 it seems pretty early that there are at the risk of oversimplifying two different types of players

01:16:36.600 through the players who, when they win, it's because of how well they've done things when they lose,

01:16:43.620 it's because of bad luck. And that's basically the narrative. And you seem, and again, I don't know

01:16:50.400 how quickly you came to this understanding. But at some point, you seem to come to the understanding

01:16:54.960 of, wait a minute, sometimes I'm winning when I make the wrong bet. And sometimes I'm losing when

01:17:01.680 I make the right bet. And that last one, by the way, must be a hard pill to swallow.

01:17:05.800 You end up sort of in your own mind formulating a very common tool we use today, which is this

01:17:11.180 decision matrix of, did I use the right process? Yes or no. Did I get the right outcome? Yes or no.

01:17:18.820 That's a lovely two by two, but it's one that you have two squares of that two by two never get

01:17:24.820 talked about, which are the ones that are at odds with each other. Let me really dig into something

01:17:30.060 you just said. So I agree that we're not seeing the whole two by two. Where I'm going to quibble

01:17:36.060 is this. Correct me if I'm wrong, but I think what you said was you could have made really good

01:17:42.160 decisions and had a good situation and ended up having a bad outcome. And that's the really hard

01:17:47.900 pill to swallow. And I actually think it's the reverse to me. I think that's the story you tell

01:17:53.740 when you're opening your book, which bringing it back to the England Patriots is when the tables are

01:17:57.900 turned, right? It's the Pete Carroll pass call on the two yard line or whatever in the Superbowl that

01:18:05.240 gets intercepted, which again, by all metrics, if you looked at this, like you were a robot looking at

01:18:11.760 reams of data, the outcome of that call was positive. That should have been a touchdown.

01:18:17.900 Right. That's a question of resulting. When we're evaluating other people's decisions,

01:18:22.500 particularly people that we don't compare to, particularly in a situation where the decision

01:18:26.440 is complicated, we have this simplifier, the shortcut called resulting. So what we do is we say,

01:18:34.240 if I know what the quality of the outcome is, I can work backwards from that to the quality of the

01:18:39.120 decision. So this is what happens to Pete Carroll. It's the last play of Superbowl 2015.

01:18:44.060 Everybody expects he's down by four to the Patriots. As always, it's always the Patriots

01:18:50.340 back to the beginning of the conversation. At any rate, everybody thinks he's going to hand the

01:18:54.000 ball off to Marshawn Lynch. Instead, he has Russell Wilson, the quarterback pass the ball.

01:18:58.460 This is very unexpected. The math is very, very complicated here. There's some game theory in

01:19:04.480 there. There's options theory, all sorts of things that people are interested in reading about it.

01:19:08.400 They can go look at Benjamin Morris on FiveThirtyEight. Suffice it to say, it is a good play.

01:19:12.800 We'll link to all of this in our show notes, by the way, because we love nerding out on this exact

01:19:16.480 sort of background. So he passes the ball. It's intercepted. They lose the Superbowl this way.

01:19:23.160 Malcolm Butler, my hero. Just, I mean. I can't imagine. Can you imagine what I was doing to my TV?

01:19:30.260 I remember looking at my daughter going, earmuffs. And it was like, fuck, yeah. I mean, I am screaming.

01:19:40.000 My wife's like, all right, I'm going to take the kids out. I'm like, yeah, yeah, yeah, that's fine.

01:19:42.900 Yeah, exactly. So this is what we do. The thing about understanding the quality of the decision,

01:19:47.720 it's complex. It's really hard. You need to know a whole bunch of different things. You need to know,

01:19:54.000 given the decision that was made, what were the possible outcomes? How often would those outcomes

01:20:00.760 occur? If you think about what are the consequences of each of those outcomes? How much do I prefer

01:20:05.740 each of those outcomes? So I can understand what the probability of getting a preferred outcome is.

01:20:09.900 If I have a outcome that I don't prefer, like an interception, like how often does that happen? And

01:20:14.660 then you also have to compare it, do that exact process with the other possibilities, the other

01:20:20.060 decisions that you could have made, and now actually compare to see how do I get the highest

01:20:24.340 expected value, the highest chance of actually winning the Super Bowl. And it's all very complicated

01:20:28.920 and involves what's the interception rate, which people don't know off the top of their head. Or

01:20:34.580 you have to think about if there's only 26 seconds left, how do I get three plays off instead of two?

01:20:39.740 And it's very hard. Suffice it to say, if you pass the ball either on the first or the second play

01:20:45.840 there, because it's second down, 26 seconds left, one time out. If you pass the ball on either the

01:20:51.160 first or second play, it means that you can do that plus hand the ball off to Marshawn Lynch.

01:20:58.120 If all you do is hand the ball off to Marshawn Lynch twice, you don't get that third play.

01:21:02.580 That third play is incredibly valuable. And the cost of the third play is whatever the

01:21:07.220 interception rate is. And the interception rate there is less than 2%.

01:21:10.380 And there's a time management piece to it as well. A pass that is not caught is an immediate time

01:21:16.500 stop. You could have a bizarre scenario under which you hand it off to Marshawn Lynch.

01:21:21.340 It turns into a bit of a scrum. You have to burn one of your timeouts.

01:21:24.820 Well, you're going to have to no matter what.

01:21:26.320 You only have one left, if I recall.

01:21:27.960 Right. So you have one left. So that's why if you hand it off to Marshawn Lynch

01:21:32.540 on the first two plays, assuming the first one fails, that's the end of your timeout. So

01:21:37.940 because you can get the clock to naturally stop on a pass, that's why if you pass on the first or

01:21:43.620 second play, you can get three plays instead of two. But that's all very complex. So what we tend

01:21:49.060 to do in those situations is we say, that's all really hard. And I don't know what any of that is.

01:21:54.800 And it's all very opaque to me. So if I understand what the quality of the outcome is, was it good or

01:22:00.880 bad in this particular case, it was disastrous. Then I know what the quality of the decision is. So

01:22:05.740 that's what's happening in that Pete Carroll thing. That's called resulting. Resulting is a heuristic

01:22:10.060 simplifier. Works really well in chess. If I lose to you in chess, I played worse than you. Works very

01:22:16.340 poorly in poker. If I lose to you in poker, who knows? And certainly very poorly in football. We

01:22:22.700 don't really know, depending on how the play worked out, whether it was a good or bad decision. But what's

01:22:28.040 interesting is that people don't do this resulting thing in situations where they feel like the quality

01:22:35.460 of the decision is known. And I say feel like, because- In quotes, yeah. Right. Because they

01:22:40.660 don't necessarily know it. But remember that what I said is resulting is a simplifier. The quality of

01:22:45.700 the decision, it's complicated. It's hard to figure out. So what happens when we feel like we do know the

01:22:49.980 quality of the decision. So I'll give you an example. If I come home one day and I say to you,

01:22:57.900 I got in a car accident. Do you know if it's my fault or not? No clue. And you won't use the fact

01:23:04.900 that I got in a car accident to work backwards like you do with Pete Carroll, because you understand

01:23:11.160 that there are certain things that really are knowable about driving. And you can ask me what those

01:23:16.060 are. Did you break the rules of the road? Were you drinking? You can ask me a whole bunch of things.

01:23:22.240 So there's all sorts of places like that where we feel like we understand what the decision quality

01:23:27.380 is. And so if we have a bad outcome, we actually tolerate it. This occurs all the time in business

01:23:32.440 as well. Right. If somebody makes a decision, let's say they execute on a decision that was made

01:23:38.100 in a committee where the whole team agreed that this was the right way to go. And then it doesn't

01:23:47.200 work out. People aren't like idiot. But if they do something on their own, that is not something that

01:23:55.520 everybody always does, then all of a sudden, if it doesn't work out, they're really under the gun.

01:24:01.180 So we have this issue of transparency that occurs. So now let me get back to this roundabout way.

01:24:07.600 So first of all, let me just give you a good example of the transparency problem.

01:24:11.280 Okay. Bear with me on this thought experiment for a second. You don't have ways. It's broken.

01:24:17.340 And you're driving to the airport with your spouse. Obviously, you have to get there in time. And you get

01:24:25.060 in the car and you go the normal way that you go, that you've always gone. And something happens,

01:24:31.400 there's a big accident on the road, and your car is not moving. And you get to the airport,

01:24:37.220 and you just missed your flight. Are you getting blamed for that? Is that your fault?

01:24:41.200 Is your spouse mad at you?

01:24:42.640 No, not in my case. I can imagine in some people's cases.

01:24:45.140 But mostly, no, it doesn't feel like it. Okay, so now you get in your car. Again,

01:24:48.620 you don't have ways, whatever your phone's broken. You get in your car,

01:24:51.640 and you announce to your spouse, I have a shortcut.

01:24:53.880 But I was just talking to Morgan over there, and they told me about this great shortcut. So I'm

01:25:00.980 going to take the shortcut because I've checked it out. I looked on a map. It's better. And now

01:25:05.540 you go to the airport, and there is an accident. Car is not moving. You make it to the airport,

01:25:10.660 and you've just missed your flight. Is your spouse mad at you?

01:25:14.420 Again, mine would not be because she's superhuman. But yes, I think many people in that situation are

01:25:20.320 absolutely getting yelled at. By the way, that's a superhuman spouse. It's like,

01:25:23.960 I cannot believe your stupid shortcut. So this is one of those cases where,

01:25:28.220 so we have this come up all the time. And when it comes up in so many different ways,

01:25:33.640 it comes up in medicine all the time. Because that's such a great example, right? If Marshawn

01:25:38.080 Lynch had been handed the ball, and he had fumbled the ball, same exact outcome. People would be pissed

01:25:44.260 at Lynch. Nobody would be pissed at Pete Carroll. That is exactly right. And they might not even be

01:25:48.620 pissed at Marshawn Lynch. They might just think that the Patriots line is too good.

01:25:52.940 Yeah, the safety, or whoever came and stripped the ball out was just exceptional.

01:25:56.760 Right. And why is that? What's the difference? Because in both cases, you fail.

01:26:00.100 Expected versus not expected.

01:26:01.700 Expected versus not expected. And then this comes up, I guarantee you this comes up in your business

01:26:05.580 all the time. It comes up in your personal life all the time. It's one of the reasons why,

01:26:10.920 for example, if someone's really unhappy in their job, they won't be aggressive enough about going and

01:26:17.200 finding another position. And the reason is that the unhappiness in their job is now sort of become

01:26:23.440 expected. Like they're not really blaming themselves for that. But if they go and find a

01:26:26.820 new position, and they're not happy there, they're going to feel like it's their fault. As an example,

01:26:30.860 this happens in business strategy. This is the way that teams end up operating, so on and so forth.

01:26:35.800 Okay, so this becomes a really big problem. So every human being understands this difference between

01:26:40.960 kind of transparent and opaque decisions. Now, here's my question for you. What decisions do you

01:26:46.120 think are the most transparent to you? Your own. These have the most transparency to us.

01:26:53.120 So what happens with our own decisions is that we allow uncertainty to bubble to the surface as the

01:27:01.220 explanation for a bad outcome more often than we would for somebody else's decisions. Why? Because

01:27:07.540 we feel like we know what the quality of our decision is, number one. And number two, we don't

01:27:12.040 really want to question the quality of our decision. So you can think about that sort of in general,

01:27:16.340 do you hand it off to Marshawn Lynch or do you pass it, is a question of when are we allowing

01:27:20.520 uncertainty as the explanation for a poor outcome. So it's innovative, unexpected, opaque, all of those

01:27:28.220 things. We don't allow the uncertainty in the door. But when we feel like we know, because obviously

01:27:33.140 you're supposed to hand it off to Marshawn Lynch, now we let uncertainty into the door. We do this for

01:27:37.280 ourselves as well. So now this brings me full circle to this thing about when we think about

01:27:42.240 this matrix and you say, well, you might make a really good decision and have a bad outcome.

01:27:48.960 That's the most painful, you said. By the way, that's just my subjective overlay. I don't know

01:27:54.280 that there's any decision-making expert out there that would argue one way or the other.

01:27:58.080 Well, I think that people do think that. So I just want to sort of pick at something. So let's just

01:28:01.920 start again. So what you said was there are these two quadrants, right?

01:28:05.540 That we would think of as discordant.

01:28:07.260 Right. So one of them is you make a bad decision.

01:28:10.960 But you get the right outcome.

01:28:12.220 You get the right outcome. And then you said even more painful is you make a good decision

01:28:16.660 and you get a bad outcome. Now, what I think is interesting is that I would actually argue

01:28:21.360 that that doesn't have a lot of pain associated with it. And let me explain why. Imagine you're

01:28:27.820 looking at this in the aftermath of the outcome. Okay. So in one case, you've gotten a good outcome.

01:28:32.200 In one case, you've gotten a bad outcome. So let me ask you to do the thought experiment.

01:28:36.320 Where do you want to dig in more? If you get a good outcome, are you looking to dig in to try

01:28:42.200 to find out if you made a bad decision that resulted in that good outcome? Or are you just

01:28:46.160 being like, good outcome? Yay me. But if you have a bad outcome, are you eager?

01:28:51.340 Honestly, I think it depends. Like using sports as an example, I think is a different example

01:28:56.660 versus in medicine. So I've talked about this before on the podcast. In medicine, there's a,

01:29:01.700 or at least in surgery, there's a conference that's typically done called the morbidity and

01:29:05.720 mortality conference, where you, as the name suggests, every week you discuss all the morbidities

01:29:12.160 and all the mortalities of the previous week. And so that is an objective definition. A person

01:29:19.300 can basically arrive dead in the hospital from, let's say a gunshot wound. They still get presented

01:29:26.260 in the conference, even though, let's say you did nothing. They literally showed up in the emergency

01:29:31.200 room. You did CPR for a minute. They were all basically already dead in the call time of death.

01:29:34.560 That still shows up, even though there's no blame to be assigned within the confines of how

01:29:38.860 medicine was practiced. So those ones don't get a lot of discussion. The ones that get a lot of

01:29:43.160 discussion are you made a decision that seemed like the right decision at the time, and it seems

01:29:50.060 defensible, but the outcome was bad. Now, of course, as you've alluded to, these are so complicated

01:29:56.100 because there's still the luck component. There's all these other things that go into it. So I guess

01:30:00.480 if your question is, where do you learn the most versus what causes the most consternation? Those

01:30:06.800 aren't necessarily the same, are they? The point that I'm making is not so much about where do you learn

01:30:10.580 the most? Because I think you learn from all four quadrants, and it's really important to learn

01:30:15.500 from all four quadrants, right? I made a good decision. I got a good outcome. I'd like to

01:30:18.720 understand when I have made a good decision that has a really good expected value, and I want to be

01:30:25.260 able to see that. I want to understand when I made a bad decision that had a bad outcome and where I had

01:30:29.780 negative expected value, and then I can understand that so I can adjust my decision making going

01:30:33.640 forward. And likewise, I'd like to understand when I made a bad decision that had a good outcome and vice

01:30:37.520 versa. What you just described is actually the whole shebag in terms of the reinforcement of this

01:30:44.380 bad kind of thinking. Because what I'm guessing does not occur every week is the, wow, this patient

01:30:50.780 did way better than expected. But we made the wrong decision. Let's have a conference about that. So let

01:30:56.240 me give you a super simple example to try to get at this, because I think that this is actually a really

01:31:00.360 big problem, particularly as I feel like now people have now understood the importance of process

01:31:08.960 language. So if we think about any business movie from the 1980s, they're like, I'm results driven,

01:31:16.320 right? Now, now you sort of look back at that, and you're a little horrified, because I think we all

01:31:20.180 understand that we don't want to be results oriented. We kind of get that that's a problem. And we'd all

01:31:25.160 like to aspire to be Sam Hinckley and trust the process. And we know we're supposed to say process,

01:31:29.500 process, process, process. But here's the problem. So now let's do this as a two by two. Okay, you have

01:31:35.820 a status quo decision, the expected decision, a consensus decision, and you have a good outcome.

01:31:45.480 Is anybody hailing you as the biggest genius of all time? No, but they're saying good job. You have a

01:31:52.040 status quo decision that has a bad outcome. Is anybody excoriating you? No,

01:31:59.080 they're saying tough luck. What could you do? You have an unexpected, an opaque, a non status quo,

01:32:07.080 a non consensus decision that has a good outcome. And they're hailing you as the biggest genius of

01:32:14.080 all time. You have a non status quo decision that has a bad outcome. And everybody's calling you an

01:32:20.660 idiot. So people understand this. So now let's think about what does that do to somebody's decision

01:32:26.620 making? Well, first of all, they want to avoid the bad outcome. This is where loss aversion comes in,

01:32:32.220 right? We don't want to be the idiot. That's the one quadrant that you want to avoid, right? Well, in

01:32:36.440 general, we want to avoid the whole right side of the table because the right side of the table contains

01:32:42.280 the idiot section. So let's think about how can we avoid the whole right side of the table? Well, one way

01:32:48.620 we can avoid sort of the bad outcomes altogether is to kind of fall down on the not deciding thing,

01:32:55.960 just sort of not doing anything that's going to cause a big loss, or a big win. Either you could do

01:33:03.480 it by omission, you could sort of not act, you could make very, very low volatility choices, there's

01:33:09.440 things that you could do there to play these strategies. The other thing that you can do, so you can

01:33:14.820 avoid deciding at all, or you can make sure that you're making things that don't really create big

01:33:20.540 losses, so you're not going to end up in the room. But the other way to deal with that whole right side

01:33:25.920 is to focus on the lower right quadrant and say, I never want to be the idiot in the room. And so

01:33:31.380 therefore, I'm going to make status quo decisions. So let's think about this. For example, it doesn't

01:33:37.460 matter if 80% or so of ear infections are viral. If a patient comes into my office, I'm giving them

01:33:43.740 antibiotics because I don't want to get yelled at. If this happens to be the time, I want to be able

01:33:48.720 to cover my butt. And that way nobody can yell at me. So now let's think about this. So what ends up

01:33:54.920 happening is that we want people to be thinking, I just want to make the best decisions. And I don't

01:34:01.320 really want to be afraid of the bad outcomes. I particularly don't want people to be afraid of

01:34:06.980 this idiot quadrant, because we would like people to be innovative. And we'd like them to be thinking

01:34:12.080 outside the box. And we'd like them to sort of push against what the status quo is, because that's how

01:34:16.140 obviously we move forward as a society, as a business, as an individual. But this is what

01:34:22.740 happens. Everybody has a morbidity conference. So even if in the morbidity conference, you're talking

01:34:29.300 about process, you're in the room because of morbidity. So everybody knows I get in the room because of

01:34:35.820 something bad. So here's a simple thought experiment that I use. Imagine that we're investing in

01:34:41.800 real estate, like we have a real estate investing group. And we have a model of the market, obviously.

01:34:46.720 And we have limited resources that we can deploy. And so we invest in a particular project, and it ends

01:34:54.500 up doing 15% worse than we expected. We're all in a room. Now, we might be in that room talking about

01:35:02.300 our model. True. We might be in the room talking about our decision process. But we're having a long

01:35:09.220 meeting in a room about the fact that the project did 15% worse than we expected. Now we do the exact

01:35:15.480 same thing. We invest in this particular project, and it does 15% better than expected. Is the same

01:35:20.980 conversation happening? No. So I don't care how much the word process comes out of your mouth.

01:35:26.980 Everybody knows they're supposed to be afraid of bad outcomes then. And here's the really big problem.

01:35:32.420 And it's true, we can talk about it in terms of medicine as well. But this particular thought

01:35:36.300 experiment that has to do with the real estate, it matters just as much as if you've overestimated

01:35:41.360 the market, which is how you'd end up 15% on the downside, than if you've underestimated the market.

01:35:47.200 Because your goal is to efficiently allocate your resources within the market that you're

01:35:53.140 allocating the resources in. And if your model's wrong, I don't care if it's wrong, if it's

01:35:58.460 underestimating or overestimating, that's a problem for efficient allocation of resources.

01:36:02.360 Not only that, but when it's 15% to the good, it's a signal that there may be risk that you

01:36:08.780 didn't identify. So for example, you could have the mean right, but you may have the variant.

01:36:14.820 That's a really big problem, because that has really big implications for what percentage of

01:36:19.500 your bankroll, or of the total money that you have, the total capital that you have that you're

01:36:24.100 supposed to deploy to that particular project.

01:36:26.500 That's a very important point, by the way, which doesn't, I think, get talked about enough,

01:36:29.800 even in that circle, which is people are very fixated on return on capital,

01:36:34.480 but they generally aren't as appreciative of risk-adjusted return on capital. So Rayrock

01:36:39.360 versus Rock is this concept that seems to be missing from the way a lot of people think about that.

01:36:44.960 So it's good that you brought that point up, because there could be a number of problems with

01:36:49.080 that issue, not just maybe you didn't allocate enough, maybe you allocated the right amount or too

01:36:54.120 much, it didn't appreciate what was at risk.

01:36:55.820 Right, exactly. So we want to be seeing those signals, right? Because we'd like to understand

01:37:00.400 it if we underestimated the vol. And in both cases, your model could be totally right. And you can just

01:37:05.400 be a couple of standard deviations away, or there's a variety of reasons on that particular time that

01:37:11.060 it could have come in low or high, that don't have anything to do with your model. But you kind of

01:37:15.460 want to dig in either way, because it's something unexpected happened. The other thing, by the way,

01:37:19.920 that happens is that when you have a downside event, and you do get in the room, people rarely

01:37:25.800 asked, should we have done worse? They asked, could we have done better? And now again, you're

01:37:30.380 reinforcing for people, I care when things go badly, when things go badly, I want to avoid

01:37:35.640 them. Now poker happens to be this very pure place, where should I have done worse, regardless

01:37:42.260 of whether I won or lost is actually a very important question. Because sometimes you have

01:37:47.720 a hand, I'm thinking about what you have. And let's say that I put you in a range where you're

01:37:53.300 very weak. And I feel that I'm very strong in comparison to you. So because you're weak,

01:37:58.420 I'm playing the hand, we would call it slow or small, where I'm...

01:38:03.140 You're trying to draw as much out as possible.

01:38:04.760 Right, which means that I'm not betting really big. Now at the end of the hand, you hit a lucky

01:38:08.280 card at the end of the hand, the very last card, I end up losing the hand. But when we turn the

01:38:12.820 hands over, it turns out that you were much stronger than I thought. In this case, I should have

01:38:18.260 lost more. Because yes, you hit a lucky card, but I should have been extracting much more money from

01:38:24.700 you because you would have tolerated it had I had your hand right. Now it may turn out that there

01:38:28.980 was no way for me to get your hand right. But I'm supposed to explore that. Because now what I know

01:38:33.960 is that if I knew what your cards were, I would have lost a lot more money than I did. That I actually

01:38:39.520 played the hand very poorly for what the hand was that you actually had. I actually underplayed my hand.

01:38:45.220 And that story, that example you use, I think to me is one of the best examples of why the title of

01:38:50.920 the book makes sense, which is... This isn't a book about poker, actually. It's a book that uses

01:38:56.900 poker as a model system. So for example, let's talk about biology. Why do we do so much biology in

01:39:03.140 C. elegans, a worm, or in this type of a mouse or this yeast? Do we care that much about mice and

01:39:09.800 yeast and worms? No. They are model systems that allow us to do things in cleaner ways.

01:39:17.660 And I think that's sort of the beauty of poker, which for the listener, they might think they're

01:39:22.000 talking about poker so much. Why? Well, I think you have to understand enough about poker to understand

01:39:26.440 what you just said. Because what you just said is, it's not often in life we will get that much clarity

01:39:32.700 after the fact, but we can still bring that level of critical thought to our decisions. To me, that

01:39:39.600 is the... If you remember nothing else from this podcast, this is the part I'd want someone to

01:39:44.880 remember. I think of poker as the C. elegans of decision-making in life. That's going to be my...

01:39:50.060 I love that. That's great. And we can think about this on the winning side as well, right?

01:39:54.240 Like, I could win a hand from you where I really should have won a lot less. Again, if I had had

01:40:03.800 perfect information, I shouldn't have actually won as much as I did. Because, for example, I could

01:40:09.120 misread your hand, and I could play a hand really strongly, and then I can be the one who hits a

01:40:13.680 lucky card. And actually, if I had perfect information, I would not have ever put that much

01:40:19.640 money at risk. So we want to explore whether the outcome is good or bad. We want to have equanimity

01:40:25.680 toward the direction that we're exploring. Could I have gotten even more out of you? Could I have

01:40:30.200 done even better? Could I have done worse? And the other thing besides that direction, by the way,

01:40:34.160 is we want to think orthogonally. Like, maybe you lost for reasons that had nothing to do with your

01:40:38.080 decision process, or maybe you won for reasons that you totally didn't predict. This happens in

01:40:43.220 financial markets all the time. You invest in a, say, a company because you think that a particular

01:40:49.940 product of theirs is going to do gangbusters or something, and then that product fails, but they

01:40:55.480 happen to acquire another company. By the way, don't take credit for that if that wasn't in your model,

01:41:00.100 but you happen to have won. So we want to be thinking up, down, orthogonal in all these directions.

01:41:05.400 So now here becomes the problem. We know that people don't like to be in this bottom white

01:41:11.040 chronic, they don't like to be in this place of, okay, something really bad happened. And if they

01:41:16.720 do get in that something really bad happened place, they want to be able to say, but it's not

01:41:22.180 my fault. It's a bad luck situation. It's like, well, my decision process was good, and everybody

01:41:26.900 had consensus, and I handed the ball off to Marshawn Lynch. What could I do? That was what I was

01:41:31.520 supposed to do. So you're driving people into this place where they're very likely to be trying to

01:41:37.140 build consensus, not necessarily of the good kind, right, of the false kind. They may be using data

01:41:42.360 not to find the truth, but to tell a data story that supports them in case they happen to get into

01:41:47.700 the room. So we really like to say the data told us so. They may just be doing what everybody has

01:41:52.640 always done. They may be slow to decide because they need buy-in or the right kind of sign-off too

01:41:57.920 often, or they need a higher level of certainty than they really should in order to make the

01:42:02.300 decision. All sorts of bad things come out of this, and what really ends up coming out of it is

01:42:06.520 that people tend to move slowly. They tend to default toward omissions versus commissions. In

01:42:11.940 other words, not deciding versus deciding, and they tend not to like to innovate too much.

01:42:17.160 Now, my guess is there's a different propensity for this across different fields. So for example,

01:42:22.720 have you ever had a chance to talk to Pete Carroll?

01:42:24.640 I haven't.

01:42:25.480 Just thinking about everything you said, it would be very interesting to have that discussion with him

01:42:29.140 and say, it'd be hard for him to do this, but go back in time to before you made that decision.

01:42:34.220 I'm pretty sure the only thing Pete Carroll was thinking about was winning. I can't imagine there

01:42:39.800 was even an ounce of thought in his mind that said, whatever decision I make, I want to make sure it's

01:42:46.140 defensible at the press conference. I just can't imagine he would have thought that, right?

01:42:49.400 I agree, and I think that's what makes Pete Carroll such a great coach.

01:42:52.440 I was just about to say, but wouldn't you see more Pete Carroll-like thinking in something like

01:42:56.900 sports, or is it less about sports and more about being the head coach versus the defensive

01:43:02.920 coordinator who, in the end, is still making a very important decision, but answers to a coach.

01:43:09.480 Of course, the coach answers to the owner, but I'm not sure I'm articulating the question well.

01:43:13.440 No, I actually think that you are. So first of all, we know across the NFL that the slowness to

01:43:17.780 adopt the analytics has been remarkable. And that's because in this particular case,

01:43:23.560 the people who are judging you are going to be like the GMs or the fans or that kind of thing.

01:43:29.800 And if they're judging you, they're going to be judging you for the unusual decisions.

01:43:34.200 So I think actually, if we go to Belichick, we can kind of see, if we go back to this idea of this

01:43:40.940 feeling of transparency, people are forgiving when there's a feeling of transparency. So if you're

01:43:46.480 Pete Carroll, at that point in his career, people are much more likely, and remember, even with him,

01:43:53.520 they really bashed him, but they're much more likely to sort of give you the benefit of the doubt.

01:43:57.680 So I actually saw a great talk by Toby Moskowitz. He's a really interesting guy. Does a lot of stuff

01:44:03.300 in sports analytics and quant and this kind of thing. And he was talking about Belichick. And he

01:44:09.100 said, when you look at his decision making, when he was with the Browns, it was dismal compared to the

01:44:13.940 analytics. And it's not actually until he wins two Super Bowls that you start to see this decision

01:44:19.960 making that's really starting to align with the analytics. It's really starting to be like,

01:44:25.400 I don't care if nobody's going to understand this decision. I'm Bill Belichick, and that's the

01:44:30.400 decision I'm going to make. But it's after two Super Bowls that he gets there. Why? Because the fans

01:44:35.200 are now tolerant of it. They're willing to tolerate the stuff that they don't understand. Why did you go

01:44:40.140 for it on fourth and five there? That's insane. No, well, because you're Belichick. So what we want

01:44:46.200 to think about is, when we think about these, what you said, the morbidity conference, what is that

01:44:52.620 encouraging in the people that you're trying to lead? You're encouraging the behavior that they

01:44:59.260 naturally bend toward, which is, I want to stay out of this lower right quadrant. So I don't want

01:45:07.040 anything bad to happen to me. But if it does happen to me, I'm going to make sure there's so

01:45:12.300 much CYA there that it's going to be okay, that nobody's going to sit there and say, I made a crazy

01:45:17.500 decision. And you're doing that because you're not, you're not exploring both directions, but

01:45:22.780 particularly because you're not getting in the room when something unexpectedly good happens. And here's

01:45:27.380 the issue. When something unexpectedly good happens, there's so much to learn from that.

01:45:32.320 Is that different, by the way, than when something good happens that shouldn't have happened based on

01:45:38.240 what people didn't know? Now I'm getting sort of Rumsfeld-y in a bit, but okay, I'm making this up.

01:45:43.560 So this could be complete nonsense. So I'm hoping you'll be able to come up with a real example.

01:45:46.820 Remember the whole Y2K thing? Everybody thought, oh my God, this could be a free for all, blah,

01:45:52.760 blah, blah. Turned out to be the biggest nothing burger in the history of civilization. Like I have

01:45:55.900 friends in college whose entire job after college was getting ready for Y2K. So nothing happened. Fine.

01:46:02.320 Now there's two ways nothing happened. One is nothing happened because there was really

01:46:06.040 nothing going on and we just didn't understand that. And we did a whole bunch of stuff to be

01:46:10.300 prepared for it. And in the end, it didn't matter. There's another thing, which is, oh no,

01:46:14.000 it was a real problem. And all of that work that was done to make sure airplanes didn't fall out of

01:46:18.460 the sky, it worked. Those are kind of different. And I'm using the problem there is there's work

01:46:23.040 involved versus decision being made. So maybe this isn't a great example. I guess what I'm trying to get at

01:46:26.900 is you can have great outcomes because there was going to be a great outcome regardless of your

01:46:33.300 decision. And then you can have great outcomes where you actually made the wrong decision and

01:46:38.060 you got the right outcome. And then there's, you made the right decision and the right outcome.

01:46:41.940 I'm almost wondering, like, are there other historical examples in wars? I'm trying to think

01:46:45.600 of like D-Day. Was that invasion on that day the right thing to do? I mean, history will forever be

01:46:50.980 changed as a result of it in the right direction. But has anyone ever gone back and said, well,

01:46:55.240 you know what? Based on the weather pattern, like Eisenhower got lucky. That shouldn't have

01:46:59.840 worked that day. And that was the wrong decision. They should have waited a day or something to that

01:47:03.300 effect. Yes. So number one, with Y2K, there's a fourth thing that you can say. So I agree

01:47:09.000 with all three things that you pointed out. It could have been good decision, good outcome. It could have

01:47:14.480 been, it would have happened no matter what we did. And so we were irrelevant. It could be that we had a

01:47:20.520 good outcome and it was all sort of a waste of time. So it was a bad decision to do anything.

01:47:24.040 It could be it doesn't really matter. Because when you sort of look at what the range of

01:47:28.200 possibilities were given the unknown information, the downside was so bad that you were actually

01:47:33.980 supposed to protect against that regardless. And were you going to waste a whole bunch of human

01:47:38.400 productivity doing it? Well, it's not really a waste because it just acts as a hedge against

01:47:42.240 just in case it's the worst case scenario. And particularly with something like that, we would

01:47:46.180 want to protect ourselves against that. But unless you dig into the win, you don't find that stuff

01:47:52.060 out. And I think that that's where the problem is. So with the Eisenhower example, what you said,

01:47:58.020 it's such a great model for how do we think about our own outcomes in our own lives. So this is what

01:48:03.480 tends to happen to us. And this is why I was saying, oh, it's actually not painful at all when you have a

01:48:09.140 bad outcome to go in and look. It's what we naturally do. Something horrible happens to me. I'm looking in

01:48:15.680 there. I'm trying to find all the luck in there. I'm exploring what were my other options? Was there

01:48:20.460 something else that could have worked out better? How often was that decision going to work out

01:48:25.080 poorly? And I'm so excited. I'm so excited to go in and discover that I didn't do anything wrong.

01:48:30.940 I'm looking for that when it's a bad outcome, when it's a good outcome. When we land on the beaches

01:48:37.120 of Normandy, and it's a pivot point in the war, and we win, people aren't looking as hard to try to

01:48:45.040 figure out to really explore the counterfactuals. Well, what if I had done this? What if I had done this?

01:48:50.460 What if it had turned out this way? So one of the first steps to being a really good decision

01:48:56.800 maker, because we have this asymmetry in how willing we are to explore these outcomes.

01:49:01.100 And it's what you just said. It's we're all living our lives having a morbidity conference.

01:49:06.280 Now, the morbidity conference that we're having in our head is specifically to go find the luck in

01:49:10.740 the whole thing. But that's the only conference we're having. We're never having the, my life is

01:49:16.240 great conference. Let me go figure out if that's because of my own decision making, or because of

01:49:20.880 luck, or if there was a better way, or I could have actually opened up a whole other set of

01:49:24.500 outcomes. Like nobody's, nobody's doing that.

01:49:27.120 That is such an interesting point. You are so correct in that. And as you're telling that story,

01:49:31.800 I'm thinking of a sort of silly example in my life. But it's one that anyone who knows me knows

01:49:37.020 I take very seriously, which is how much I love archery. And I shoot a lot. So when I am not in New York,

01:49:43.100 obviously, when I'm back in San Diego, I'm shooting constantly. And I'm always trying to push the

01:49:47.660 limit of my capabilities, which means you're losing arrows. You're just invariably, if you're 60,

01:49:53.720 70 yards away, trying to hit something the size of a softball, you're going to miss every once in a

01:49:58.340 while. I'm generally losing an arrow to the tune of one a day. Now, these arrows are very expensive.

01:50:03.280 So this is becoming quite a problem. Every one I've missed, I can tell you a hundred reasons why,

01:50:09.740 and not bad luck. I can tell you the mistake I've made. I don't think I've once ever given thought

01:50:17.440 to the perfect shots. I don't think I've ever spent time thinking about that ever. I think

01:50:24.060 90% of my energy goes into thinking about the 10% of the shots I didn't like. And I thought that's

01:50:30.560 the right way to do it. Isn't that making me a better shooter? What I'm hearing you say is that's

01:50:34.800 part of making you a better shooter.

01:50:36.080 It's also going to encourage, I mean, if you think about sort of societally, if it's the bad

01:50:40.460 outcomes that are triggering the dives, first of all, you're losing half of the opportunities to

01:50:44.940 learn.

01:50:45.240 If not more than half, because you might even be making more positive outcomes than less.

01:50:49.620 Right, right. Well, let's assume that the chances of having an unexpectedly good outcome are symmetric.

01:50:55.360 And let's make it that the thing that triggers us doing a dive is that it's unexpected. And you can

01:51:02.640 decide if it's like a one standard deviation away or the standard deviation and a half or whatever it

01:51:07.660 is, we can decide and that would obviously depend on the uncertainty and how much ball there is and

01:51:12.180 so on and so forth. But we'd figure that out. And let's assume it's symmetric. So by only digging into

01:51:16.780 the downside, we're doing two things. One is we're losing half of our opportunities to learn.

01:51:20.380 And the second thing is that we're reinforcing the risk aversion. We're saying, hey, what we really

01:51:26.620 care about is downside outcomes. So you better avoid those. And three is we're completely quashing

01:51:32.120 anybody ever doing anything unexpected, where, because if you do something unexpected, the chances

01:51:37.320 that somebody is going to allow for luck or that you yourself are going to allow for luck is the

01:51:41.040 explanation is just going to kind of go poof. And we don't like that. So there's all sorts of bad

01:51:45.360 stuff that comes out of it. So when we get an outcome, we want to basically do what you were

01:51:50.480 just doing with the beaches of Normandy, which is to say, here's the outcome. Before I make any

01:51:56.980 conclusions about it, let me think about what are the other possible things that could have occurred.

01:52:03.120 So I have to think about what are the things that didn't happen but could have those counterfactuals.

01:52:08.360 And then I need to think about what was my preference for those things? How much did I like each of

01:52:13.860 those things? And how often were those things going to happen? So it's just basically thinking

01:52:18.060 about what was the payoff for those things. Obviously, you can compare those to cost. But

01:52:21.380 let's just simplify it. What were the possibilities? What were my preferences for those possibilities?

01:52:27.920 And what was the probability of each of those possibilities occurring? And then, so that's the

01:52:33.720 first step with the beaches of Normandy. It ended up being a particular way. But given that they decided

01:52:39.260 to do what they did on that particular day, what are the other ways that it could have turned out

01:52:43.340 with what probability would have that have turned out? And how much would we have liked or those

01:52:47.840 things depending on what our goal was, which in this case was to win the war. And then now we can

01:52:52.040 take that and we can iterate it and say, what if they had waited two days? What if they had done it

01:52:58.400 two days before? What if they had done it in a different way? They had gone from land instead of sea.

01:53:04.620 Now we can start thinking about if they had chosen other options, then how would that have compared

01:53:10.840 in terms of what the expected value was for the option that they actually chose? And you can do

01:53:16.460 this for anything. So you can do this for the archery. I assume that a bullseye is pretty

01:53:19.720 unexpected. At certain ranges, it's more surprising than it is not. Right. What you can say is you could

01:53:24.520 actually go in and start forecasting it. I'm going in at a particular range, given what I think that

01:53:28.840 my skill level is, what percentage of the shots do I think I'm going to get a bullseye? So forecast it.

01:53:34.260 Say this is the percentage of times that I think I'm going to get a bullseye. Now, if it's a particular

01:53:38.240 day and you're well below that, you would say, hmm, I wonder what happened today. What was the

01:53:43.740 situation today where I was well below what my expectation was for the number of bullseyes?

01:53:48.520 But now, if you're way above, you do the exact same thing. And this is what we don't do. I can

01:53:55.160 guarantee you, because I know this is the way that people think. If you think, oh, I don't know how many

01:53:59.480 times you shoot at the target. I've done so much of the work you're describing without the important

01:54:03.900 piece. But at 50 yards, for example, I know, just based on what I do, that I should be able to put

01:54:10.580 8 out of 10 inside of a certain diameter. So I spend a lot of time thinking about when I'm hitting

01:54:18.220 6 out of 10 inside of that range. What's going on? But there's days I'm hitting 10 out of 10.

01:54:22.700 This is what you do. You walk off and you dance. You're like, look at me.

01:54:25.840 Yeah, I don't think about why.

01:54:26.880 You're like, whoa, check me out. I'm great. And the thing is that, of course, you may have

01:54:33.500 stepped up in skill, but wouldn't you like to know exactly what the adjustment, what was it that

01:54:37.380 actually clicked for you that day? What is it that changed?

01:54:40.000 And I know it's not that I stepped up in skill because it can vanish the next day. So it's clearly

01:54:44.300 something that is transient.

01:54:46.160 Well, one of the reasons why it may be transient...

01:54:48.500 Is because I haven't identified it and marked it and incorporated it.

01:54:51.740 Right, because it could be totally transient. It could be that you get to a certain skill

01:54:55.480 level and this is going to be your range. It's going to be 6 to 10.

01:54:58.160 Oh, well, by transient, I just mean what I'm really saying is I'm sure that I did something

01:55:02.580 better on that day, but it's transient because I didn't identify it and make it a part of my

01:55:07.480 permanent... It could be that if we could look at me under the 6 out of 10 day versus the 10 out of 10

01:55:12.400 day. I mean, I'm positive I'm doing something different. There is no way equipment, wind, or any

01:55:18.660 other factor other than me explains that. But by only comparing 8 to 6, not 6 to 10,

01:55:25.340 I'm missing a greater disparity, which in theory should call out more obvious deltas. It's not

01:55:32.040 just that you're missing half the time, it's you're using half the discrimination. It's like it's easier

01:55:36.680 to see that there are two lines if they're a foot apart versus a millimeter apart.

01:55:40.980 So if I had to say, like if someone asked me what's the single biggest factor between a player who

01:55:48.620 doesn't make a ton of progress or makes slower progress compared to a player who really,

01:55:54.920 really makes a lot of progress and becomes elite in poker, I would say that's it. That when you

01:56:00.340 listen to someone like Eric Seidel talk about poker, you would hardly know whether they won or

01:56:06.800 lost the hand. All they're doing is exploring what are the counterfactuals? Like what are the other

01:56:12.440 lines that I could have chosen? Did I have the person's hand right? What if I had bet a little bit

01:56:17.300 more? What if I had bet a little bit less? What if I hadn't played the hand at all? What would have

01:56:22.000 happened if I played this hand that I didn't play? And you would hardly know. Like if you listen to

01:56:26.020 them, you would have no idea. In fact, probably you would assume they had lost. Right, because it's

01:56:29.800 so foreign to hear somebody. Because they're exploring so much. You'd be like, whoa, you must have

01:56:34.440 really lost that tournament. And they'd be like, no, I won the tournament. And you move on. And what you

01:56:40.020 hear from players who aren't making that kind of progress is it goes two ways. When they talk about

01:56:44.740 the hands they won, it's just sort of like I was great. And they don't really spend a lot of time

01:56:48.720 on it. When they talk about the hands they lost, it's a lot of exploration of the luck elements.

01:56:53.780 And the thing about poker, because the luck element in a given instance, like in the short run is so

01:56:58.940 prominent, that it's so easy to focus on the bad card that came. So you'll have conversations with

01:57:08.220 people where they'll talk about like, oh, I had this hand. And this person, it's called sucking out.

01:57:14.620 They sucked out on me, meaning that they happened to hit a good card. And to somebody who's thinking,

01:57:20.420 who spent a lot of time around someone like Eric Seidel, you're looking at it and saying,

01:57:24.920 well, you should have won that hand way beforehand. Or maybe you shouldn't have played that hand at all.

01:57:29.120 So for example, like if I have a particular hand where you end up hitting a really good card,

01:57:33.800 it may be that I should have made you fold before then. Now sometimes it's you were going to be

01:57:38.420 sticky, you were going to stay around, I had mathematically way the best of it. And whatever,

01:57:43.440 bad luck happened. But sometimes it's I should have made you fold a lot earlier,

01:57:48.200 I should have made you pay more for the privilege of hitting that card, that actually the way that

01:57:54.220 I bet the hand meant that you were making money to my bet when I could have made you lose money to

01:57:58.880 my bet, regardless of whether you were going to beat me on that particular hand in the long run,

01:58:02.480 I'd like to make you make losing decisions. Maybe I shouldn't have played the hand at all.

01:58:07.280 There's all sorts of other things that you can be exploring in those situations,

01:58:10.880 and they're not getting explored. Instead, what's getting focused on is I can't believe

01:58:14.460 that person sucked out on me.

01:58:16.080 So I have this framework for learning things that require skill. And I see a parallel now that

01:58:21.180 I've never seen before. So my framework, and I use swimming as the analogy for this,

01:58:25.320 because I learned to swim as an adult. So this was really the first time I explored this framework.

01:58:29.580 But I do think it applies to many things, languages, you pick it, frankly.

01:58:33.500 So the framework is we start out as unconsciously incompetent. So you take a person like me at 31 who

01:58:42.140 doesn't know how to swim, you put them in the water. There's no ambiguity about how bad I am.

01:58:46.660 But I don't know why. All I can tell you is I can't swim. I can't actually tell you why I can't swim.

01:58:51.840 So the first step of learning how to swim is becoming consciously incompetent. You actually have

01:58:57.140 to understand why you're really bad at this. Oh, once you start to understand Archimedes principle,

01:59:04.040 that's like a very important part of understanding how to swim. The more of you that's underwater,

01:59:08.560 the more buoyancy there is to hold you up. Once you start to understand how Bernoulli works,

01:59:14.260 oh, all of a sudden, there's this difference about the velocity of a fluid over a body, etc.

01:59:19.360 And then by the way, you're no better as a swimmer in those two boxes, right? The difference

01:59:23.120 between... Right. I just want to know the percentage of people who are learning swimming

01:59:27.140 who cite Archimedes and Bernoulli. I just love that that's the way that you're... I don't know if I

01:59:33.340 still have them, but I would love it if I did. I used to journal so relentlessly about swimming.

01:59:38.200 I had reams of journals that I would write in every single day of my free body diagrams of my body in

01:59:46.960 different swimming positions and trying to figure out ways to make this less bad. When I got into this,

01:59:53.120 I became so obsessed that I swam twice a day, every day. This model makes so much sense to me.

01:59:58.520 Well, you don't actually start swimming until you reach the third box, which is now you are

02:00:03.400 consciously competent. Now, the problem is you can't do that for long. So you tend to vacillate a lot

02:00:10.440 between the second and third box, which is consciously competent. And then you go back to being

02:00:15.640 consciously incompetent. But as long as you can maintain focus, you will vacillate between those.

02:00:21.340 And usually it's your physical fatigue that will push you from the third box back to the second

02:00:26.920 box. Now, when you become a really good swimmer, like Michael Phelps, you transcend to this fourth

02:00:32.540 box. You are now unconsciously competent. Everything becomes sort of autonomic at that level. Now,

02:00:39.500 thinking about this through decisions, as you were telling the story about Eric, I'm thinking about

02:00:43.640 the first level. So let's just call it four levels. Now, the first level is you only examine losses

02:00:49.760 through the lens of luck. So that's our default. I would argue that takes no effort, just as being

02:00:55.800 unconsciously incompetent is our default. Layer two or level two thinking is I only examine my losses

02:01:04.480 through the lens of luck and skill. That's great.

02:01:08.800 So you're taking some responsibility, but it's only triggered by the loss.

02:01:11.620 Only triggered by the loss. Then you have a level three, which is I'll do that,

02:01:15.720 but I will also now examine my wins through skill. And then the fourth level, which I'm guessing is

02:01:22.720 exactly where someone like Eric is, is no, I'm going to look at wins and losses through luck

02:01:28.780 and skill.

02:01:29.840 Yeah. So what I would say is I think that the thing about examining wins, I do think that examining

02:01:38.580 wins through the lens of skill is pretty natural. We don't spend a lot of time on it. It's kind of an

02:01:44.660 assumption of skill. I would put wins down in level one. Let's put it this way. I would focus

02:01:49.680 more on, I would actually think about it, not just in terms of, are you examining the quadrants that

02:01:58.260 are discordant in a symmetric way? So what I would say is that on the level one thinking is I lean on

02:02:07.440 the discordance on the loss side and I lean on the concordance on the win side. And I just kind

02:02:13.660 of do that naturally. And I don't spend a lot of time thinking about either of them.

02:02:17.520 That makes sense.

02:02:18.480 I don't spend a lot of time thinking about either of them. Then I think that level two is you're still

02:02:24.620 leaning on concordance on the win side. You're not examining those very much, but you do start to

02:02:28.940 take the losses.

02:02:29.040 At least you examine the discordant losses.

02:02:31.020 Right. You examine the concordant losses now. So you examine the losses that are actually due to

02:02:36.040 your own undoing. Then I'll get to the fourth level. Then I think that level three is I'm looking

02:02:45.980 in all four quadrants. So I'm looking at wins that are due to skill, that are due to my good

02:02:51.340 decision making. I'm looking at wins that are due to my bad decision making. I'm looking at losses

02:02:56.540 that are due to my bad decision making. I'm looking at losses that are due to my good decision making.

02:03:01.280 And I'm trying to, I'm really trying to sort of look at those all equally.

02:03:04.760 Where I think the fourth level comes in is when it's concordant on the win side that

02:03:13.420 you don't accept it.

02:03:15.120 In other words, that you say, I made good decisions and I won, but it does not mean

02:03:20.960 there wasn't a better way.

02:03:22.660 So let me actually start to really clean up in there and say, it's not about I made a

02:03:30.040 decision.

02:03:30.380 It was a winning decision.

02:03:31.740 I was positive expected value for it.

02:03:34.380 And so I'm done.

02:03:36.180 It's now I so care that I got onto what we would call the primary line of play that I

02:03:43.000 really came up with the best possible solution that I could given what I knew that I won't

02:03:48.320 be satisfied with just finding out that there was concordance.

02:03:51.040 Like I want to go beyond that on the win side.

02:03:53.040 I think what we're going to do in the show notes is literally take everything you just

02:03:56.280 said and turn it into a printable one pager that every one of us should carry with us.

02:04:01.680 Because that's that is a very elegant way to think about that.

02:04:06.220 I promise you I've never done what you just said.

02:04:08.900 Like I've never once had a level four thought in my life.

02:04:12.040 Well, first of all, let me just say that I only thought about it because you gave me

02:04:16.080 the levels.

02:04:16.580 But it's that last bit.

02:04:18.380 So right now I'm writing a workbook, very close to finished with it, where in chapter

02:04:23.620 three, I really dig into that piece, which is when you have a good outcome and it turns

02:04:30.520 out it's from good decisions, what are you doing with that?

02:04:33.400 Are you done?

02:04:34.540 Or are you trying to figure out if there was even better one beyond that?

02:04:38.320 And here's the reason why we don't like to do that.

02:04:40.460 Because when we think about what sort of creates the fabric of our identity, our beliefs are

02:04:47.760 the threads that our identity is woven out of.

02:04:50.380 By doing a level four thought, you risk they're undermining.

02:04:52.860 Tearing a hole, tearing a hole in it.

02:04:54.820 Because one of the things that we want to do is feel like that fabric is strong, like

02:04:59.420 our beliefs are good and we're competent.

02:05:02.740 I mean, this idea of competence and being able to bank credit for the way that things have

02:05:09.060 turned out in our life is so core to what forms our identity that when you don't stop

02:05:15.740 at, hey, my decisions were good and I had a good outcome, when you actually dig around

02:05:21.220 in there, you're risking now turning something that was a win into a loss.

02:05:27.140 So this is where I love your idea of the levels.

02:05:30.140 So if we think about that level one, like I won, it must be good.

02:05:33.040 Why wouldn't you go further than that and say, well, maybe I won because of bad luck.

02:05:36.600 You're turning a win into a loss.

02:05:38.280 And in this particular case, we're taking win as the outcome.

02:05:41.120 I had a good outcome and you sort of want to just stop there because you don't want

02:05:44.900 to dig down and find out you might have made a bad decision because now you turned a win

02:05:48.960 into a loss.

02:05:49.880 So that's where when you get to level two, you're willing to do that.

02:05:53.380 You're willing to say, I want a long run win.

02:05:56.340 And what that means is I might have to turn this outcome that was a win into a loss.

02:05:59.980 When you get to level four, what you say is, okay, I had a good outcome.

02:06:04.780 I looked and I saw that my decision making was pretty good.

02:06:07.520 But now I'm willing to turn that win into a loss in order to get long run better.

02:06:12.160 You were basically completely exposed at that point.

02:06:15.060 I can imagine how your book became immediate fodder for every hedge fund manager and so

02:06:20.820 forth.

02:06:21.020 Because again, I think the advantage of that type of investing bears some resemblance to

02:06:26.680 poker, which is you can't really hide your decisions.

02:06:30.040 Your outcomes are very clear and which is not to say there's not a huge element of luck just

02:06:35.300 as there is with poker, but it's basically an industry where your decisions are exposed.

02:06:40.520 Every decision you make, in the end, you're going to be able to trace it to an outcome.

02:06:45.300 And frankly, unless you're not in the business of making money, you have to be wed to good

02:06:50.140 outcomes.

02:06:50.900 So the poker player wants to do well.

02:06:52.920 The hedge fund manager wants to do well.

02:06:54.680 There's nowhere to hide, I guess, is what I'm trying to get at.

02:06:56.940 I agree in theory that there's nowhere to hide.

02:07:00.080 The interesting thing that I think is how long people can hide from it.

02:07:06.300 That's what I find so fascinating is you look at poker and you're like, you're getting answers

02:07:11.720 right away, right?

02:07:12.640 Like you win or lose.

02:07:14.060 So here's the problem.

02:07:15.300 It's like, okay, at the end of the year, somebody lost, right?

02:07:19.040 But why?

02:07:20.220 How far are you willing to go to say, I just got bad luck?

02:07:24.540 That's the question.

02:07:26.020 And I'm just really fascinated that people can go really far.

02:07:31.040 They can really convince themselves that they just got bad luck.

02:07:34.960 And I think that that's true.

02:07:37.240 Obviously, if you're running a hedge fund, you have investors that you're accountable

02:07:40.740 to.

02:07:41.540 But we all know of hedge fund managers where they had a couple of good years and then

02:07:46.400 people are sticking with them.

02:07:47.620 It's like you have a couple of good years to start.

02:07:49.440 There's this path dependence to this.

02:07:51.020 And it's true of poker.

02:07:51.620 If a poker player comes out of the gate in the first six months and loses everything

02:07:55.080 in sight, they're very unlikely if they continue to lose.

02:07:58.620 They're going to say, oh, I must be really bad at this.

02:08:00.820 But for every poker player who comes out of the gate with a terrible first six months,

02:08:04.500 there's a poker player.

02:08:05.700 I actually remember way back when, a long time ago, there was a new player who came

02:08:11.340 on the scene and they won a World Series of Poker bracelet.

02:08:15.880 I mean, they were really young.

02:08:16.880 I mean, early 20s.

02:08:18.080 And nobody had ever heard of them before.

02:08:19.840 And they won.

02:08:21.080 And I thought, oh, you know, that's nice.

02:08:22.280 Because that happens, right?

02:08:23.180 Obviously.

02:08:23.920 Then they won a second one.

02:08:25.360 And I was like, oh, that might be just the ruin of them.

02:08:28.720 Now, why would I say that?

02:08:29.700 Like, they had won a second World Series of Poker bracelet.

02:08:31.900 Because I understood this path dependence, right?

02:08:34.580 That if you come out of the gate and you do really well, your ability to fool yourself

02:08:40.200 afterwards for a very long time.

02:08:42.040 It becomes infinite.

02:08:43.080 It becomes almost infinite.

02:08:44.040 And you see that in the hedge fund world all the time.

02:08:46.360 And obviously, if an investor comes along and they don't show any results, by the way,

02:08:50.500 sometimes unfairly, it becomes very hard for them to recover from that.

02:08:54.400 Sometimes unfairly, because maybe it just went against them for that year or whatever.

02:08:58.720 But when it's the reverse, when they come out of the gate strong, you can fool yourself

02:09:03.240 for a really, really, really, really long time.

02:09:05.460 And that's the thing that I find so fascinating.

02:09:07.660 That's what really kind of lights me up.

02:09:10.800 And trying to figure out the capacity for self-deception is so a bottomless well.

02:09:17.460 Is there an industry that you look at where you just see, on average, it is more amenable

02:09:23.440 to higher level thinking the way we just described the four levels in poker, like this framework?

02:09:30.020 Is there a place where, either through some externality of the way the industry is set up

02:09:35.460 and the rate of time being short between decision and outcome and some sort of transparency,

02:09:41.380 that we have a space where there's a greater incentive to rise to that level?

02:09:45.660 First of all, let me just say this, that obviously, we're only talking about places

02:09:49.880 where, in the shorter run, luck is going to have a strong influence.

02:09:53.620 So when you look at sports that are really highly under the influence of skill, people

02:10:00.520 are getting selected out.

02:10:01.920 Nobody's fooling themselves.

02:10:03.860 Well, actually, it was just bad luck that I didn't get to be a professional NFL linebacker.

02:10:09.780 I mean, it might be if you got an injury, but I'm 5'5 and I weigh 130.

02:10:14.540 Such bad luck that I wasn't.

02:10:16.340 People are just like, no, I just wasn't good enough.

02:10:17.980 So we know that in cases where it's very transparent that if you're not doing well,

02:10:23.980 we can trace it back to skill.

02:10:25.540 So the more chess-like we are, the less the capacity for us to fool ourselves.

02:10:30.580 If I lose to you in chess, it's very hard for me to say that I got bad luck because,

02:10:34.620 I don't know, where's the luck?

02:10:35.700 I don't know, go find it for me.

02:10:37.060 There's a little bit of luck, but not much.

02:10:39.220 So the more on the chess range of things we are, the less the capacity to do that.

02:10:44.100 So first of all, we've got to be in an environment where there's a strong short-run influence

02:10:47.840 of luck.

02:10:49.040 Now, the tighter the feedback loop, the more feedback you're getting, the more that somebody

02:10:55.600 needs to rise to the occasion.

02:10:57.240 So if you were to look at what's the capacity, which individuals are going to thrive more

02:11:01.880 in those systems, definitely the ones who are much more willing to dig in and dig around.

02:11:07.680 So people who are in higher frequency trading, for example, where those feedback loops or the

02:11:12.720 time horizons are shorter, I think are going to be more likely to be thinking in this way

02:11:17.820 and if they're investing their own money.

02:11:21.040 Ah, that's a very interesting point.

02:11:22.600 It's got to, Nassim, right?

02:11:23.900 You got to have skin in the game.

02:11:25.020 Are there poker players, by the way, that play with other people's money?

02:11:27.360 And by that, I mean not money that they won from somebody else, but I see.

02:11:31.340 Okay.

02:11:31.600 Yeah.

02:11:31.840 So there's backing situations.

02:11:33.160 There's backing.

02:11:33.340 Okay.

02:11:33.440 Yeah.

02:11:33.640 There's the same thing.

02:11:34.540 The less you're trading on sort of reputation and the more that you're trading on your own,

02:11:38.260 your own dollars.

02:11:39.300 Also, I think that you're more likely to end up in a spot where you're thinking that way.

02:11:42.900 So basically you have tons of skin in the game.

02:11:45.660 There's a high element of luck and skill and the feedback loop is short.

02:11:50.720 Those are three situations that will tend to push you towards either abject failure or getting

02:11:57.900 better, moving up the ladder of this thinking.

02:12:00.180 It's a little chicken and egg.

02:12:02.760 So I don't know.

02:12:03.960 Is it the people who succeed are more naturally, kind of more naturally tend to that kind of

02:12:09.320 thinking and obviously it gets reinforced or.

02:12:13.200 What would Eric do if he wasn't the best poker player in the world?

02:12:16.660 Well, I know what he would do because he was an options trader.

02:12:19.720 Okay.

02:12:20.260 So again, options trading, you could argue that it's sort of similar, right?

02:12:23.820 It's incredibly similar.

02:12:24.980 It's almost the same.

02:12:26.040 But would he go and be a good CEO of a blue chip company?

02:12:30.380 Like if he all of a sudden tomorrow, assume he had the domain knowledge.

02:12:33.880 Okay.

02:12:34.120 So he could go in and be the CEO of General Electric or Home Depot or Exxon Mobil or something

02:12:40.100 where you could put a chip in his head that gives him the domain expertise to understand

02:12:44.340 exploration and production or whatever.

02:12:47.360 But could you move that industry or that company within its industry relative to its peers by bringing

02:12:54.420 that layer of thinking?

02:12:56.160 Yes.

02:12:56.500 So that's what, that's, that's kind of what I want to say.

02:12:58.760 Is there a little bit of a chicken and an egg issue?

02:13:01.000 Because I have met people across industries where there are people who are thinking this

02:13:05.560 way, where there are people who are really hungry for it.

02:13:07.840 Now, in terms of this idea of like, what are you doing when you find out it was pretty

02:13:11.820 good decisions and good outcomes, or are you only doing postmortems and not thinking

02:13:16.740 about it in terms of expectancy?

02:13:18.420 That's a problem that you see across all industries.

02:13:20.580 But also across industries, you see people who are really hungry to try to incorporate

02:13:25.540 that into their solution space.

02:13:28.640 How am I dealing with this?

02:13:29.660 How am I running my teams?

02:13:31.420 So if you just take the investing world, I see people who think that way all the way

02:13:36.480 from people who are trading options to people who are doing seed round venture capital, much

02:13:43.340 different than late stage venture capital where you might, the time horizon now is faster.

02:13:48.120 I mean, not the same speed as options, but if you're like in the seed round, we're talking

02:13:52.540 horizons, it could be a decade.

02:13:54.340 There are fund managers who are in long short funds where they might put on seven positions

02:13:59.380 in a year.

02:14:00.180 You're hardly getting any data.

02:14:01.860 And they're, they're really looking to think this way because what's interesting is that

02:14:07.040 while trading options might foster this kind of thinking better because you're getting

02:14:13.500 feedback more quickly, while you might end up getting sort of selected out by the people

02:14:18.720 who you're accountable to quicker, while maybe that particular type of mind would tend

02:14:23.700 to get drawn toward, say, options trading better.

02:14:27.240 The issue is that in absence of the world telling you quickly, this type of thinking actually

02:14:33.260 becomes much more important.

02:14:35.180 Yeah, it becomes more important and more difficult.

02:14:37.240 Right.

02:14:37.480 Because when you spread the feedback loops out now, the way that I always think about

02:14:42.660 it is uncertainty gives you the leeway to allow bias to come in.

02:14:49.100 And that's the difference between the chess and poker.

02:14:51.140 In chess, because you've taken a lot of uncertainty away, you don't have the leeway to fool yourself.

02:14:55.780 That is an absolute gold point.

02:14:57.300 But in poker, you do.

02:14:58.200 So we can sort of think about if you look at the difference between, say, trading options

02:15:01.520 and something that has a much longer time horizon, the longer the time horizon, the more capacity

02:15:06.620 you have, the more leeway the world is giving you to fool yourself.

02:15:11.740 So then this kind of thinking, really thinking in this systematic way, really modeling the

02:15:17.040 world in this way becomes much, much, much more important.

02:15:20.180 It becomes a bigger imperative for you to be poking, for you to be thinking counterfactually,

02:15:25.060 for you to have people challenging your ideas, for you to be trying to think about, am I doing

02:15:31.340 this because it's status quo?

02:15:34.200 And if it doesn't go well, I won't get blamed.

02:15:37.700 Am I innovating enough?

02:15:38.900 Am I allowing people to challenge my ideas?

02:15:40.900 Am I challenging my ideas myself enough?

02:15:43.760 Am I looking at that discordant box that has to do with good outcomes?

02:15:50.180 Or am I only willing to look at the discord when it's a bad outcome?

02:15:53.720 When the bad outcome is concordant, am I like super excited about that?

02:15:57.840 Or am I imagining, well, maybe there was a better way?

02:16:00.880 How am I digging into all of those spots?

02:16:03.220 And then if you're in a leadership role, how am I propagating or am I not propagating these

02:16:08.760 problems across the team?

02:16:10.600 Am I telling the team through my actions that they should be terrified of bad results?

02:16:15.480 As I'm thinking about the way that the world is giving me the leeway to be able to sort of

02:16:19.500 have my beliefs lead me around on a leash, am I infecting my team with that same problem?

02:16:26.440 Because now I can be creating this whole other problem.

02:16:29.180 And it's such a long time horizon that maybe the world's going to tell me that this was

02:16:33.280 actually problematic.

02:16:34.860 You should be digging into that as if you're getting the answer tomorrow, as if you're playing

02:16:39.060 chess and you're going to find out right away.

02:16:40.980 And you should be doing everything you can to try to make sure that you don't find out

02:16:46.640 you hung your rook.

02:16:48.040 This to me is why swimming is harder than riding a bike.

02:16:50.460 You can teach anybody to ride a bike relatively quickly for a couple of reasons.

02:16:53.640 One, the feedback loop is very quick.

02:16:56.700 Like anytime you're out of balance, I mean, we just naturally internally, our inner ears

02:17:01.060 allow us to sense that disequilibrium instantaneously.

02:17:05.120 So you get the fastest feedback loop imaginable and the consequences are quite high.

02:17:10.180 It hurts significantly when you're not in balance.

02:17:12.720 Contrary to popular belief, swimming is about balance.

02:17:15.040 It's the exact same thing.

02:17:16.220 You just happen to be in a medium that is so much more dense than air that the feedback

02:17:20.920 loop is much slower and the consequences are much more gentle.

02:17:25.100 It doesn't hurt when you're out of balance.

02:17:27.640 In fact, if you fell every time on a bicycle that you were out of balance while learning

02:17:31.840 to swim, you'd never ride a bike again.

02:17:33.680 It would, you'd be so black and blue.

02:17:35.520 So in that sense, swimming is gentler, meaning you have more excuses.

02:17:40.420 It's longer time horizon, but it's that much more important that you do the examination.

02:17:45.760 So if anybody can mindlessly ride a bike very soon after being shown how to do it, you can't

02:17:51.720 mindlessly swim for years after learning how to swim.

02:17:55.320 It's that hard.

02:17:56.700 Swimming then becomes the example of where it's more important that you bring that level

02:18:02.240 of thinking if you want to make any progress.

02:18:04.960 I agree.

02:18:05.680 And let me add something to this, which correct me if I'm wrong, but I think this is in the

02:18:10.280 same space as you're thinking.

02:18:11.440 Sort of thinking about the soft landing issue, the wider the skill gap between you and the

02:18:21.100 people that you're competing against, there's so much more cushion for a soft landing.

02:18:26.680 So this is something that I sort of think about is by not exploring that, sure, I made winning

02:18:34.140 decisions and I won.

02:18:35.720 By not willing to go beyond that.

02:18:37.820 Imagine if you're in a situation where, let's say we're in some sort of zero-sum game, so

02:18:42.840 we're trading against each other.

02:18:44.900 And if I were making perfect decisions against you, every time that we put a dollar on the

02:18:52.280 table, I would get a quarter.

02:18:54.600 But because of the decisions that I'm making, every time we put a dollar on the table, I'm

02:18:58.740 getting a nickel.

02:18:59.860 So if we think about those quadrants, right, when I look at I have a good outcome, I won a

02:19:03.520 nickel, and my decision-making was good because I was positive expectancy, right?

02:19:08.360 I was winning a nickel.

02:19:09.500 I was winning 5%.

02:19:10.640 But by not exploring, I don't see the 20 cents that's sitting there that I could get every

02:19:17.340 single time.

02:19:18.500 And that's the problem with a soft landing.

02:19:20.900 That's the problem with having that cushion, is it doesn't let you know that there's something

02:19:26.700 better that you could be doing.

02:19:27.960 I know we got to wrap because we both have to get moving, but there's one topic that

02:19:31.800 you very loosely, I think we were going to go down the path of it by name, but then we

02:19:36.800 didn't.

02:19:37.360 But I want to bring it up because it's such an important concept in my world.

02:19:41.100 And it's something I have done for such a long time, but never had a great name for it.

02:19:46.780 And when I read your book, I was like, oh, awesome.

02:19:49.740 I'm going to be plagiarizing this all day long.

02:19:52.600 And it's the idea of backcasting.

02:19:54.400 Oh, yeah.

02:19:54.940 A moment ago, you used the term forecasting.

02:19:57.260 Right.

02:19:57.540 We're all about forecasting.

02:19:59.000 Well, I want to be doing this in Q4, and therefore I need to do this today and this tomorrow and

02:20:04.580 this tomorrow.

02:20:05.200 And I'll tell you how I use it, and then I'll...

02:20:08.040 And let me just say for the record, backcasting is not a word that I coin.

02:20:11.900 Oh, all right.

02:20:12.760 I don't want to take credit for that.

02:20:13.920 I've got to reference you who references someone else.

02:20:15.140 You can say you read about it in my book.

02:20:16.860 I didn't invent that word.

02:20:18.080 All right.

02:20:18.340 Thank you for clarifying that.

02:20:19.780 So prior to reading your book, I would just refer to it as reverse engineering, but that never really

02:20:24.400 had any resonance with people.

02:20:25.620 But now when I say backcasting, it forces me to explain the contrast between backcasting

02:20:30.260 and forecasting, whereas reverse engineering doesn't have a striking of a contrast.

02:20:35.080 So the place that I use it is in understanding the exoskeleton of the body.

02:20:40.080 So the actual physical structural demise of ourselves.

02:20:44.640 So we're all sitting here thinking about exercise and everybody knows they should exercise, but

02:20:48.420 nobody really can give you a great reason for why.

02:20:50.460 We've been sitting here talking about the outliers, professional athletes, but outside of people

02:20:54.760 who get paid to do something with their body, what are the rest of us doing?

02:20:58.940 It's amazing how often you push people on this.

02:21:00.980 They can't give you a great answer.

02:21:02.280 And because I ask everybody under the sun, this question, I'll tell you all answers basically

02:21:06.200 fall into these three buckets.

02:21:07.520 So if you're asking someone who's not getting paid to do a sport, why do you exercise?

02:21:13.020 90% of the answers are, I like the way it makes me look.

02:21:16.420 I like the way it makes me feel.

02:21:19.280 And I like the freedom and flexibility it gives me with what I eat.

02:21:23.440 Some combination of those three.

02:21:24.440 There are other answers that show up a lot less frequently, such as I love the social interaction

02:21:28.160 that comes with it and things like that.

02:21:29.320 Now, I would argue all of those are fine reasons to exercise, but I don't think they're

02:21:33.360 good enough.

02:21:33.960 The good enough one only comes when you start thinking about backcasting, which is what

02:21:38.500 do you want your body to function like the day before you take your last breath?

02:21:44.100 Really think about this.

02:21:45.460 So I've taken this to an extreme and people have heard me talk about this in the podcast

02:21:48.660 before, but I come up with this idea of the centenarian Olympics, which is instead of training

02:21:54.280 for like an Olympics that's now where you would actually have to pull vault and run a hundred

02:21:58.740 meters in under 10 seconds.

02:22:00.100 Think about what an Olympic games would look like of a hundred year olds and ask yourself,

02:22:04.920 what do you want to be able to do in that Olympic game?

02:22:07.940 So I have like 18 things I want to be able to do when I'm a hundred, if I get to be a

02:22:11.440 hundred and they're all quite mundane, but they're all very pragmatic and practical.

02:22:16.180 One of them is being able to do a 30 pound goblet squat, which again, I could do today blindfolded

02:22:21.240 on one leg a hundred thousand times, but at a hundred, that's actually going to be quite

02:22:26.000 a task.

02:22:26.720 And it's important because that replicates picking up a toddler, which presumably if I'm

02:22:31.500 a hundred, I've got great grandkids.

02:22:32.940 I want to be able to pick them up.

02:22:34.120 I want to be able to stand up off the floor using no support and only a single point of

02:22:39.780 my own, meaning one arm and both my legs and get up off the floor without help.

02:22:44.620 Can I do that today?

02:22:45.500 Of course.

02:22:46.240 Can you do that today with your eyes shut?

02:22:48.700 Very few hundred year olds can do that.

02:22:50.480 So I've kind of 18 of these things I want to be able to do.

02:22:54.280 And that is backcasting.

02:22:56.120 It's going to the finish line and saying, if I can do that at a hundred, what was I able

02:23:00.920 to do at 90?

02:23:02.180 If I can do that at 90, what must've been true at 60 and before you know it, I realized,

02:23:07.700 wow, if I want to do that at a hundred, now that I'm 46, I actually have to be able to

02:23:11.620 do all of these other things.

02:23:12.540 So there's this whole matrix that comes out of it.

02:23:15.100 That's why that concept for me just resonated so much.

02:23:17.720 Cause I really, I don't think medicine spends enough time backcasting.

02:23:21.420 Yeah.

02:23:21.620 So let me try to add another benefit of backcasting.

02:23:24.140 So just to be clear, backcasting is, so when we forecast what we think is there's a goal

02:23:29.720 that I would like to reach, how do I get there?

02:23:32.000 And in general, what you're thinking are, what are the things that I'm doing in order

02:23:34.780 to get to this goal?

02:23:36.100 When you backcast, what you say is it's the day afterwards.

02:23:39.560 So I have some goal I'd like to be able to do X in a year.

02:23:43.360 And now it's a year and a day and I achieve my goal.

02:23:46.680 And what you ask now is you look back and you say, how did I actually achieve this?

02:23:52.520 And you can also do something called a premortem.

02:23:55.060 So a premortem would be, I have this goal that I'd like to achieve.

02:23:58.520 I'd like to achieve it in a year.

02:24:00.480 It's now a year and a day and I failed.

02:24:02.420 Why did I fail?

02:24:03.060 Why did I fail?

02:24:04.040 Now, here's an added benefit that I would like to offer to you.

02:24:07.680 When we forecast, when we think about, here's my goal, how do I get there?

02:24:12.740 We're thinking about the things we're doing.

02:24:15.300 Here are the decisions that I make along the way.

02:24:17.860 So we're thinking about all the things that are within our decision-making power, within

02:24:21.560 our control, that are in the sort of skill side of the equation.

02:24:24.880 But what happens when you backcast and you say, I actually succeeded, how did I get here?

02:24:30.320 But particularly, what happens when you do a premortem and you say, I failed, how did

02:24:34.560 I get here?

02:24:35.100 Now, you allow luck into the equation.

02:24:39.400 So as an example, if I'm thinking about when I'm 100, I would like to be able to do a 30

02:24:46.920 pound goblet squat, which I've never heard before, but now I'll say that because I learned

02:24:51.300 that.

02:24:52.080 But I'm now 100.

02:24:53.480 It's the day I've turned 100 and I cannot do that thing.

02:24:56.220 Why?

02:24:57.160 Well, I can think of some things like I didn't actually exercise enough, but I can also think

02:25:02.280 of I tripped and I fell and I broke my hip.

02:25:05.100 Now, tripped and fell, that can happen to anybody.

02:25:08.220 Probably not a matter of skill.

02:25:10.380 It's just something can happen.

02:25:11.560 Like you could hurt yourself.

02:25:12.820 But the break my hip might have to do with my bone density.

02:25:15.560 That's going to cause you to explore other things that maybe wouldn't be as obvious.

02:25:19.420 Because you saw, nobody's thinking, you don't see that bad luck can occur if you're not

02:25:26.120 working backwards.

02:25:27.060 And then to your point, once you work backwards, you can now ask yourself some questions because

02:25:31.480 you see the luck.

02:25:33.080 You can say, okay, if it's on the good side, can I increase the chances that that thing

02:25:38.800 happens to me?

02:25:39.540 That I see is luck is going to intervene in my way.

02:25:42.400 Can I increase the chances that's going to happen for me?

02:25:45.100 Okay, so you could ask that.

02:25:45.880 But then on the downside, which is really important to explore, let's say the breaking the hip thing,

02:25:50.660 can I decrease the chances that I break my hip?

02:25:53.420 Yes.

02:25:53.800 What are the things that I could do?

02:25:54.960 If the answer is no, then you can say, are there any hedges available to me?

02:26:00.920 Can I hedge against this bad luck thing happening to me?

02:26:04.260 Sometimes the answer to that is also no.

02:26:07.380 Regardless, what you can say is, okay, I've got my hedges set up.

02:26:10.600 I'm doing what I can to decrease the chances that it occurs, but obviously it still could.

02:26:15.160 So let me think now, before that bad luck intervenes, what am I going to do in response?

02:26:20.560 Because if I plan for that in advance, I'm going to be making decisions right now in a

02:26:26.940 calmer state of mind where I'm not pants on fire, freaking out, rather than being reactive

02:26:32.660 to what the world might deliver me at the time.

02:26:35.180 So if I've already thought about it, if I already have a plan in place, I'm not wasting

02:26:39.560 time.

02:26:40.100 I'm going to be much more calm.

02:26:41.660 I'm going to think about it.

02:26:42.500 So this is one of the big difference between working forwards and working backwards is that

02:26:45.720 when we work forwards and we're thinking about how do we execute on any kind of strategic

02:26:48.960 plan, because we can think about, I want to be able to do this thing when I'm 100 and we're

02:26:53.220 developing a strategic plan for how we get there.

02:26:55.880 We really focus on what are the things that we're going to do that are going to get us

02:26:59.980 there.

02:27:00.280 And we don't say, what are the ways that luck might intervene?

02:27:04.560 But when you work backwards, you naturally start to see the luck.

02:27:07.680 And then you can start to explore all of these questions that then makes your strategic

02:27:11.520 plan much better.

02:27:12.600 It makes you explore different stuff.

02:27:14.700 And does your workbook cover exactly this type of decision tree that we just went through?

02:27:19.060 When is this coming out?

02:27:19.940 A year from now.

02:27:21.180 So I'm just finishing up the draft now.

02:27:23.940 And then you have to lay it out.

02:27:25.660 And you know how that goes with books.

02:27:26.740 Yeah, yeah.

02:27:27.180 So basically what I'm doing with the workbook is I'm taking these concepts that I talk about

02:27:31.400 broadly and sort of bigger idea way and thinking in bets.

02:27:35.380 And I'm saying, OK, how do you practically make a decision?

02:27:39.080 How do you build out a decision tree?

02:27:40.640 How do you think about the way that luck might intervene?

02:27:44.280 How do you think about your own decision making?

02:27:46.100 How do you see the luck?

02:27:47.180 How do you communicate with other people such that you're making sure that you're getting

02:27:52.680 the purest feedback from them, that you're not biasing their feedback, that you're actually

02:27:58.400 hungry for the information and you're seeking the feedback, which is, by the way, the first

02:28:02.060 step, which most of us don't do.

02:28:03.760 Most of us aren't seeking the feedback.

02:28:05.680 So when you're hungry for the feedback, you're actually thinking about what does the future

02:28:10.000 look like?

02:28:10.640 What could the outcomes be?

02:28:11.760 How am I going to react to those things?

02:28:13.880 How do I understand the sum of my experience and what that's supposed to teach me?

02:28:18.460 So it was really trying to think about how do I allow people who don't do this for a

02:28:22.980 living, who don't spend their time building out probabilistic decision trees, and this

02:28:28.040 is not the way that they look at the world.

02:28:30.020 How do I take somebody like that and show them the value and teach them how to actually

02:28:34.240 think within this framework?

02:28:35.600 Well, I think that means we're going to have to sit down and talk in about a year.

02:28:38.820 Yeah, I hope so.

02:28:39.580 When that book comes out.

02:28:40.000 I want to also just say one thing just for the listeners, because I would like you to

02:28:43.660 put it in the show notes.

02:28:44.480 So I'm going to say it right here.

02:28:45.380 What are you doing?

02:28:46.000 A lot of the stuff that we've talked about with luck and skill, I really want to recommend.

02:28:50.000 Well, first of all, on the forecasting stuff, let me just, again, recommend Super Forecasting,

02:28:53.980 which I know is going to end up in the show notes.

02:28:55.480 But there's a really, really wonderful book by Michael Malbison that's called The Success Equation.

02:29:01.640 And it's all about the influence of luck and skill in our lives.

02:29:04.460 It's got a lot of sports stuff in it, but just a lot of really digging down and drilling

02:29:09.660 down into understanding luck and skill and how you sort of understand what the difference

02:29:15.280 is between the range of activities and sort of where you can see more influence of skill,

02:29:19.460 where you can see more influence of luck, how you can sort of parse those apart.

02:29:23.660 This huge influence on our lives is luck.

02:29:27.280 And he really drills down into that in a way that if people are really interested in the

02:29:32.140 conversation that we had today, then I guarantee they would be really interested in the success

02:29:37.520 equation.

02:29:38.660 And Malbison also, I mean, you can just search him and you can see a whole bunch of videos of his

02:29:43.020 where he talks about luck and skill.

02:29:45.860 A lot of his writing talks about luck and skill and sort of separating the two and understanding

02:29:50.580 the different influences.

02:29:51.840 And yeah, I just can't recommend that book more.

02:29:54.460 We'll put together a pretty robust set of notes here, as we always do.

02:29:57.340 And it will point to everything we've spoken about.

02:29:59.660 Plus, I'm sure after the fact over the next couple of weeks, you and I will have back and

02:30:03.640 you'll think of something else that you want to hear.

02:30:05.460 So I'd love to make this the sort of the compendium for all these things.

02:30:08.180 And then, of course, I can't wait to sort of talk about this workbook.

02:30:10.340 It sounds like it's kind of an amazing tool.

02:30:12.700 I mean, again, I would say like in the last few hours, I've realized that I'm not nearly

02:30:16.980 as good a decision maker as even I think I am.

02:30:19.020 And I don't say that to be sort of arrogant.

02:30:20.820 I just think, look, I understand decisions at least as well as anybody else has been my bias.

02:30:26.500 And I'm at least as probabilistically savvy.

02:30:29.260 But I realize like there's tons of blind spots that I'm still experiencing.

02:30:32.900 And I'm not even close to approaching the potential that that a person would have to

02:30:38.180 make better decisions.

02:30:39.060 So that said, I find myself somewhat overwhelmed by how complicated this is.

02:30:44.840 I'm already thinking like Travis is going to put together some great

02:30:47.860 tear outs that will hold us over until your book comes out.

02:30:50.860 But having a book that could be sort of a guidepost to a decision matrix would be fantastic.

02:30:57.100 And let me just for people who have listened to this and felt sort of overwhelmed.

02:31:00.800 It's funny because this happens to be the chapter that I'm writing right now is that it's really,

02:31:06.280 really good to have this framework to be thinking about decisions and to really recognize the

02:31:11.240 probabilistic nature of the world and the way that things turn out for you and understanding

02:31:16.380 what the structure of a really good decision is.

02:31:18.600 That being said, that doesn't mean that you have to sit down with a slide rule for every

02:31:23.700 decision that you make.

02:31:24.640 And in fact, for the majority of decisions that you make in your life, you can actually

02:31:29.000 decide quite fast.

02:31:30.600 And what's interesting is that the way to really understand when can I decide fast is to understand

02:31:38.000 this broader framework.

02:31:39.220 This broader framework that would allow you to take the bigger decisions in your life and

02:31:42.700 actually start to think about them in this way where you're really forecasting them is

02:31:46.560 actually the way to understand when can I go really fast.

02:31:50.080 And what it will allow you to do is stop taking 15 minutes to decide what to order in a restaurant

02:31:56.180 because you have to understand this framework in order to sort of get when you can go fast

02:32:00.900 and when you can go slow.

02:32:02.060 Because when I do talk to people about this kind of framework, they do feel a little bit

02:32:05.820 overwhelmed and they're like, oh my gosh, how am I ever going to make a decision again?

02:32:09.960 Like I'm going to need a supercomputer in order to do it.

02:32:12.720 It's very few decisions that you actually need to take a ton of time on.

02:32:15.980 Mostly you can decide fast.

02:32:17.900 And on the ones that are bigger decisions, embracing the probabilistic nature actually

02:32:24.460 gets you out of the analysis paralysis.

02:32:26.440 Because what you realize is once you've identified your options and you've got an option that you

02:32:30.400 see is better relative to the other ones, you stop focusing on achieving absolute certainty

02:32:36.000 and start focusing on achieving relative to the other things that I could do.

02:32:40.260 This one definitely looks better to me.

02:32:42.440 And because I recognize that there's all sorts of stuff I don't know and there's all sorts

02:32:46.980 of different ways the world can turn out.

02:32:48.700 And as long as I'm taking the things that occur in my life and trying to learn from them

02:32:52.920 in a real way, I'm not going to be so terrified of sort of the downside.

02:32:56.840 And that really frees you up.

02:32:58.420 You've said that so, so elegantly.

02:33:00.120 And I think that reiterates this sort of the notion that intuitively people get it, right?

02:33:05.300 You go fast on the straightaway and slow on the curves.

02:33:07.420 This framework shows you where the curves are.

02:33:09.380 Yeah.

02:33:09.620 And that frees you up to go faster on the straightaways.

02:33:11.720 Yes.

02:33:12.100 And interestingly enough, it frees you up relative to what you were doing before to go faster

02:33:18.220 on the curves as well.

02:33:19.560 Absolutely.

02:33:20.360 To not try to figure out in that moment, if I don't know the exact right path to take,

02:33:26.380 I'm paralyzed and I just need to slam the brakes on.

02:33:28.940 But to recognize when, well, this path is going to be good enough.

02:33:32.620 I'll spare all the listeners the race car analogy that is like zooming into my cortex

02:33:37.660 at the moment because there is a perfect analogy to driving around a corner in a race car,

02:33:42.900 but I will spare everybody that.

02:33:44.040 You can put it in the show notes.

02:33:45.080 There we go.

02:33:45.940 Well, Annie, thanks so much.

02:33:46.720 This was really great.

02:33:47.500 I really appreciate you responding to my random request to sit down and talk about this.

02:33:52.640 And I can't wait to do it again when the new book comes out.

02:33:54.180 Well, thank you.

02:33:54.960 I'm excited to come back when the workbook is done.

02:33:56.940 But this was like an amazing conversation, very different than most of the podcasts I've

02:34:00.740 done.

02:34:00.960 So I really appreciate it.

02:34:02.160 That's the way we do things here.

02:34:03.580 We go deep.

02:34:04.580 Yeah.

02:34:04.820 You can find all of this information and more at peteratiamd.com forward slash podcast.

02:34:12.420 There you'll find the show notes, readings, and links related to this episode.

02:34:16.540 You can also find my blog at peteratiamd.com.

02:34:20.040 Maybe the simplest thing to do is to sign up for my subjectively non-lame once a week email

02:34:24.520 where I'll update you on what I've been up to, the most interesting papers I've read,

02:34:28.200 and all things related to longevity, science, performance, sleep, et cetera.

02:34:32.980 On social, you can find me on Twitter, Instagram, and Facebook, all with the ID Peter Atiyah

02:34:37.760 MD.

02:34:38.420 But usually Twitter is the best way to reach me to share your questions and comments.

02:34:42.020 Now for the obligatory disclaimer.

02:34:43.660 This podcast is for general informational purposes only and does not constitute the practice

02:34:47.820 of medicine, nursing, or other professional healthcare services, including the giving of

02:34:52.700 medical advice.

02:34:53.420 And note, no doctor-patient relationship is formed.

02:34:56.880 The use of this information and the materials linked to the podcast is at the user's own risk.

02:35:01.960 The content of this podcast is not intended to be a substitute for professional medical

02:35:05.600 advice, diagnoses, or treatment.

02:35:08.200 Users should not disregard or delay in obtaining medical advice for any medical condition they

02:35:12.460 have and should seek the assistance of their healthcare professionals for any such conditions.

02:35:17.920 Lastly, and perhaps most importantly, I take conflicts of interest very seriously for all of my

02:35:22.620 disclosures, the companies I invest in and or advise, please visit peteratiyahmd.com forward

02:35:29.320 slash about.

The Peter Attia Drive - July 01, 2019

#60 - Annie Duke, decision strategist： Poker as a model system for life—how to improve decision making, use frameworks for learning, and apply 'backcasting' to boost your odds for future success

Episode Stats

Length

Words per Minute

Word Count

Sentence Count

Misogynist Sentences

Hate Speech Sentences

Summary

Transcript