Making Sense - Sam Harris - #151 — Will We Destroy the Future？

Nick Bostrom is a Swedish-born philosopher with a background in theoretical physics, computational neuroscience, and logic, and many other interesting intersecting topics. But officially, he s a Professor of Philosophy at Oxford University, where he leads the Future of Humanity Institute, a research center focused largely on the problem of existential risk. And today we get deep into his views on existential risk by focusing on three of his papers, which I ll describe in our conversation. We talk about the Vulnerable World Hypothesis, which gets us into many interesting tangents with respect to the history of nuclear deterrence and the possible need for what he calls turnkey totalitarianism, we talk about whether we re living in a computer simulation, and the doomsday argument, which is not his, but it s one of these philosophical thought experiments that have convinced many people that we might be living close to the end of human history. And I hope you ll agree that the difference between those two scenarios is one of the more significant ones we can find. I really enjoyed talking to Nick, and I find his work fascinating and consequential, and that s a combination I find fascinating and very consequential. Make sure to check out his work! Sam Harris This is a podcast by a philosopher I ve been hoping to get on the podcast for quite some time. I m sure you'll agree that he's one of my favorite philosophers. If you like what he's doing, please consider becoming a supporter of The Making Sense Podcast by becoming a patron. It helps me out tremendously. I can t wait to hear what you're doing here. -Sam Harris Thank you for listening to the podcast, too! -Epsiode: "Making Sense" by and by . - "The Good Thing by Nick" by The Good Thing by The Bad Thing by , & so on and so on & so forth, etc., etc. Thanks for listening? -- Thank you, in the making sense? -- I mean, not too fast, I mean it's such a simple one, I know that it's good, you know that so I can be so good and so so much so that so that I can make it so good, and so I don't need to be so much of it, right right, right, so can you be that kind of thing, right? - Thank you so much?

00:00:00.000 Welcome to the Making Sense Podcast.

00:00:08.820 This is Sam Harris.

00:00:10.880 Just a note to say that if you're hearing this, you are not currently on our subscriber

00:00:14.680 feed and will only be hearing the first part of this conversation.

00:00:18.440 In order to access full episodes of the Making Sense Podcast, you'll need to subscribe at

00:00:22.720 samharris.org.

00:00:24.140 There you'll find our private RSS feed to add to your favorite podcatcher, along with

00:00:28.360 other subscriber-only content.

00:00:30.520 We don't run ads on the podcast, and therefore it's made possible entirely through the support

00:00:34.640 of our subscribers.

00:00:35.900 So if you enjoy what we're doing here, please consider becoming one.

00:00:46.820 Welcome to the Making Sense Podcast.

00:00:49.060 This is Sam Harris.

00:00:51.460 Today I am speaking with Nick Bostrom.

00:00:54.760 Nick is someone I've been hoping to get on the podcast for quite some time.

00:00:58.360 He is a Swedish-born philosopher with a background in theoretical physics and computational neuroscience

00:01:06.880 and logic and AI and many other interesting intersecting topics.

00:01:14.120 But officially he's a professor of philosophy at Oxford University, where he leads the Future

00:01:20.040 of Humanity Institute.

00:01:21.380 And this organization is a research center which is focused largely on the problem of existential

00:01:28.820 risk.

00:01:30.220 And today we get deep into his views on existential risk by focusing on three of his papers, which

00:01:38.180 I'll describe in our conversation.

00:01:39.860 We talk about what he calls the Vulnerable World Hypothesis, which gets us into many interesting

00:01:47.500 tangents with respect to the history of nuclear deterrence and the possible need for what he

00:01:55.160 calls turnkey totalitarianism.

00:01:57.880 We talk about whether we're living in a computer simulation.

00:02:00.300 He's the father of the now-famous simulation argument.

00:02:04.640 We talk about the doomsday argument, which is not his, but it's one of these philosophical

00:02:10.420 thought experiments that have convinced many people that we might be living close to the

00:02:14.340 end of human history.

00:02:16.240 We talk about the implications of there being extraterrestrial life out there in the galaxy

00:02:21.800 and many other topics, but all of it is focused on the question of whether humanity is close

00:02:29.320 to the end of its career here or near the very beginning.

00:02:34.560 And I hope you'll agree that the difference between those two scenarios is one of the more

00:02:41.360 significant ones we can find.

00:02:44.260 Anyway, I really enjoyed talking to Nick.

00:02:46.640 I find his work fascinating and very consequential, and that's a good combination.

00:02:51.800 And now I bring you Nick Bostrom.

00:03:01.040 I am here with Nick Bostrom.

00:03:02.880 Nick, thanks for coming on the podcast.

00:03:04.900 Yeah, thanks for having me.

00:03:06.520 So you are fast becoming, or not too fast, it's been years now that I've been aware of

00:03:12.260 your work, but you are becoming one of the most provocative philosophers I can think of.

00:03:18.800 And really, it's wonderful to read your work.

00:03:21.220 And I want to introduce you, but perhaps to begin with, how do you view your work and

00:03:28.440 what you focus on at this point?

00:03:30.080 How do you summarize your interests as a philosopher?

00:03:32.420 Well, that's always been a challenge for me.

00:03:35.020 Broadly speaking, I'm interested in big picture questions for humanity and figuring out which

00:03:40.300 direction is up and which is down.

00:03:42.120 That is, out of all the things you can be pushing on or pulling on in the world, which

00:03:46.920 ones would actually tend to make things better in expectation.

00:03:50.660 Yeah.

00:03:51.080 And then various kind of sub-questions that come out from that ultimate quest to figure

00:03:55.060 out which direction we should be heading in.

00:03:57.040 Yeah.

00:03:57.220 So when I think about your work, I see a concern that unifies much of it, certainly, with existential

00:04:05.780 risk.

00:04:06.380 And I don't know if this is a phrase that you have popularized or if it's just derivative

00:04:11.140 of your work, but how do you think of existential risk?

00:04:15.360 And why is it so hard for most people to care about?

00:04:19.620 It's amazing to me that this is such an esoteric concern and you really have brought it into

00:04:24.640 prominence.

00:04:25.200 Yeah, I introduced the concept in a paper I wrote back in the early 2000s, the concept

00:04:33.860 being that of a risk either to the survival of Earth originating in talent life or a risk

00:04:39.860 that could permanently and drastically reduce our potential for desirable future developments.

00:04:46.460 So in other words, something that could permanently destroy the future.

00:04:50.500 I mean, even that phrase, I mean, you really have a talent for coming up with phrases that

00:04:57.940 are arresting and, you know, it's such a simple one.

00:05:01.020 It permanently destroy the future.

00:05:03.320 You know, there are probably more people working in my local McDonald's than are thinking about

00:05:09.660 the prospect of permanently destroying the future.

00:05:13.300 How long have you been focused on this particular problem?

00:05:17.340 And again, why is it, there's something bewildering about trying to export this concern to the

00:05:24.360 rest of culture, even to very, very smart people who claim to be worried about things

00:05:30.800 like climate change.

00:05:32.340 Why is existential risk still such an esoteric concern?

00:05:36.400 Well, it's become less so over the last few years.

00:05:38.560 There is now actually a community of folk around rationalist community, the EA community,

00:05:44.680 you know, various academic centers and EA is an effect of altruism.

00:05:49.360 Yeah.

00:05:49.600 And about not, not just these, but, but kind of radiating out from these number of individuals

00:05:55.580 that, that are quite interested in this.

00:05:57.860 So I think the comparison to the McDonald's restaurant would no longer be true now.

00:06:01.860 Maybe it was true several McDonald's years ago.

00:06:04.780 Why it is?

00:06:05.160 Well, I guess you could ask that or you could ask why it's no longer the case.

00:06:08.780 I mean, I don't know that the default should be, if we're looking at academia,

00:06:11.540 that questions receive attention in proportion to their importance.

00:06:15.940 I think that's just kind of a poor model of what to expect from academic research.

00:06:20.980 So one can ask why it's changed.

00:06:23.520 I mean, on one level, you're asking people to care about the unborn, if the time horizon

00:06:28.800 is beyond their lives and the lives of their children, which, which seems on its face, probably

00:06:35.580 harder than caring about distant strangers who are currently alive.

00:06:40.900 And we know we're already pretty bad at that.

00:06:43.020 Is it, is that a major variable here?

00:06:45.980 Sure.

00:06:46.840 It's an extreme case of a public good, right?

00:06:50.100 So generally in a simple model of the market economy, public goods tend to be undersupplied

00:06:55.960 because the creator of them captures only a small fraction of the benefits.

00:07:00.580 The global public goods are normally seen as the extreme of this.

00:07:04.940 If all of humanity benefits from some activity or is harmed by some activity, as in maybe

00:07:10.540 the case of global warming or something like that, then the incentives facing the individual

00:07:14.640 producer are just very dissociated from their overall consequences.

00:07:18.940 But with existential risk, it's even more extreme, actually, because it's a transgenerational

00:07:22.740 good in the sense that all future generations are also impacted by our decisions concerning

00:07:29.300 what we do about existential risks.

00:07:31.740 And, and they are obviously not in a position in any direct way to influence our decisions.

00:07:36.160 They can't reward us if we do things that are good for them.

00:07:39.020 So if one thinks of, of, of human beings as selfish, one would expect the good of existential

00:07:45.140 risk reduction to be undersupplied.

00:07:46.820 Like you could imagine if somehow people could go back in time, that future generations would

00:07:51.660 be willing to like spend huge amounts of money to compensate us for our efforts to reduce

00:07:57.440 sex risk.

00:07:58.580 But since you can't, that, that transaction is, is not possible, then there is this undersupply.

00:08:05.300 So that, that, that, that could be one way of, of like explaining why there's relatively

00:08:11.380 little.

00:08:11.760 And there was something about what you said about it's harder to care.

00:08:16.240 Like it, it's a little strange, some, that, that caring should be something that requires

00:08:20.860 effort.

00:08:21.640 If, if one does care about it, it doesn't seem like it should be a straining thing to do.

00:08:26.100 And if one doesn't care, then it's not clear why one should be, what, what, what, what motive

00:08:30.760 one would have for trying to strain to, to start caring.

00:08:33.640 So it's a framing problem in many cases so that there's a certain set of facts, let's say the

00:08:39.780 reality of human suffering in some distant country that you have never visited, you have

00:08:45.560 no connections there.

00:08:47.040 This information can just be transmitted to you about the reality of the suffering.

00:08:52.120 And it transmitted one way you find that you don't care and transmitted another way, the

00:08:58.820 reality of it and the, and the, and the analogy to your own life or the lives of your children

00:09:03.020 can be made more salient.

00:09:05.040 And so we know that we know, you know, through the work of someone like Paul Slovic, we know

00:09:09.140 there are moral illusions here where people can be shown to care more about the fate of

00:09:14.580 one little girl who's delivered to them in the form of a personalized story.

00:09:19.800 And they'll care less about the fate of that same little girl plus her brother.

00:09:24.580 And they'll care less still if you tell them about the little girl, her brother, and the

00:09:30.760 500,000 other children who are also suffering from a famine.

00:09:35.260 And you just get this diminishment of altruistic impulse and the amount of money they're willing

00:09:41.500 to give to charity and all the rest.

00:09:43.400 And it goes in the wrong direction.

00:09:45.660 As you scale the problem, people care less.

00:09:48.160 So we know we have some moral bugs in our, in our psychology.

00:09:52.020 Yeah, so the original paper about existential risk made the point that from a certain type

00:10:00.580 of ethical theory, it looks like existential risk reduction is a very important goal.

00:10:06.740 If you have a broadly aggregative consequentialist philosophy, say if you're a utilitarian, and

00:10:12.740 if you work the numbers out, the number of possible future generations, the number of individuals

00:10:17.460 in each of those that can live very happy lives, then you multiply that together, and then

00:10:23.160 it looks like even a very small chance in the probability that we will eventually achieve

00:10:27.320 this would have a very high expected value.

00:10:30.980 In fact, a higher expected value than any other impacts that we might have in more direct ways

00:10:36.900 on the world here and now.

00:10:38.200 So that reducing existential risk by, you know, one thousandth of one percentage point would

00:10:44.000 be, from this utilitarian perspective, worth more than eliminating world hunger or curing

00:10:48.900 cancer.

00:10:49.860 Now, that, of course, says nothing about the question of whether this kind of utilitarian

00:10:55.720 perspective is correct or is agreeable to us.

00:10:59.320 But it just notes that that does seem to be an implication.

00:11:01.680 Yeah, I'm definitely a consequentialist of a certain kind, so, you know, we don't need

00:11:07.880 to argue that point.

00:11:09.000 But one thing that's interesting here, and this may be playing into it, is that there seems

00:11:13.500 to be a clear asymmetry between how we value suffering and its mitigation and how we value

00:11:23.300 the mere preemption of well-being or flourishing or positive state.

00:11:29.580 So that suffering is worse than pleasure or happiness is good.

00:11:36.520 You know, I think if you told most people, here are two scenarios for how you can spend

00:11:40.940 the rest of your day.

00:11:41.980 You can spend it as you were planning to, living within the normal bounds of human experience,

00:11:47.640 or I can give you one hour of the worst possible misery, followed by another hour of the deepest

00:11:54.640 possible happiness.

00:11:56.000 Would you like to sample the two extremes on the phenomenological continuum?

00:12:01.420 I think most people would balk at this because we think, we have a sense that suffering is,

00:12:06.280 on some level, you know, in the limit is worse than any pleasure or happiness could be.

00:12:11.900 And so we look at the prospect of, let's say, you know, curing cancer and mitigating the suffering

00:12:18.180 from that as being more important ethically than simply not allowing the door to close

00:12:26.320 on future states of creativity and insight and beauty.

00:12:32.300 Yeah, I think one might want to decompose different ways in which that intuition could

00:12:37.680 be produced.

00:12:38.240 So one might just be that for us humans, as we are currently constituted, it's a lot easier

00:12:43.600 to create pain than to create a corresponding degree of pleasure.

00:12:47.760 We might just evolutionarily be such that we have a kind of deeper bottom than we have

00:12:53.600 a height.

00:12:55.220 It might be possible, if you think about it in biological term, in a short period of time

00:13:00.000 to introduce more damage to damage reproductive fitness more than you could possibly gain within

00:13:06.160 the same amount of time.

00:13:07.120 So if we are thinking about these vast futures, you want to probably factor that out in that

00:13:13.320 you could re-engineer, say, human hedonic systems or the hedonic systems of whatever inhabitants

00:13:20.000 would exist in this future so that they would have a much larger capacity for upside.

00:13:25.040 Right.

00:13:25.420 And it's not obvious that there would be an asymmetry there.

00:13:28.060 Now, you might nevertheless think that given, in some sense, equal amounts of pleasure and

00:13:32.660 pain, and it's a little unclear exactly what the metric is here, that there would nevertheless

00:13:37.700 be some more basically ethical reason why one should place a higher priority on removing

00:13:44.540 the negative.

00:13:45.260 A lot of people have intuitions about equality, say, in economic context, where helping the

00:13:52.420 worst of is more important than further promoting the welfare of the best of.

00:13:56.700 Maybe that's the source of some of those intuitions?

00:14:00.400 Actually, there's one other variable here, I think, which is that there is no victim or

00:14:07.360 beneficiary of the consequence of closing the door to the future.

00:14:12.680 So if you ask someone, well, what would be wrong with the prospect of everyone dying painlessly

00:14:20.300 in their sleep tonight, and there are no future generations, there's no one to be bereaved by

00:14:26.780 that outcome.

00:14:27.340 There's no one suffering the pain of the loss or the pain of the deaths, even.

00:14:31.800 So people are kind of at a loss for the place where the moral injury would land.

00:14:39.420 Yeah.

00:14:39.720 So that is a distinction within utilitarian frameworks between total utilitarians who think you basically

00:14:48.340 count up all the good and subtract all the bad.

00:14:50.860 And then other views that try to take a more so-called person-affecting perspective, where

00:14:57.040 what matters is what kind of happens to people, but coming into existence is not necessarily

00:15:05.280 a benefit.

00:15:06.560 And now I would say some kinds of existential catastrophe would have a continuing population

00:15:14.880 of people who would be experiencing the bad.

00:15:16.960 If you might say the world getting locked into some totalitarian, like really dystopian

00:15:22.620 totalitarian regime, maybe there would be people living for a very long time, but just having

00:15:27.480 much less good lives than could have existed.

00:15:31.540 So in some scenarios of existential catastrophe, there would still be inhabitants there.

00:15:37.160 Yeah, no, I think it's pretty clear that destroying the future could be pretty unpleasant for people

00:15:43.860 who are along for the ride.

00:15:45.040 Now, I'd like just to harken back a few minutes ago, like on the general premise here.

00:15:51.100 So I don't see it so much as a premise, this utilitarian view.

00:15:54.720 I mean, in fact, I wouldn't really describe myself as a utilitarian.

00:15:58.780 It would be more just pointing out the consequence.

00:16:00.640 There are various views about how we should reason about ethics, and there might be other

00:16:05.020 things we care about as well, aside from ethics.

00:16:07.740 And rather than directly trying to answer, what do we have most reason to do all things

00:16:11.240 considered, you might break it down and say, well, given this particular ethical theory,

00:16:15.140 what do we have most reason to do given this other value or this other goal we might have?

00:16:19.480 And then at the end of the day, you might want to add all of that up again.

00:16:23.120 But insofar as we are trying to reason about our ethical obligations, I have kind of a

00:16:31.820 normative uncertainty over different moral frameworks.

00:16:35.760 And so the way I would try to go about making decisions from a moral point of view would be

00:16:40.600 to think I have this moral parliament model.

00:16:44.540 It's a kind of metaphor, but where you try to factor in the viewpoints of a number of different

00:16:50.680 ethical theories kind of in proportion to the degree to which you assign them probability.

00:16:55.380 It's kind of interesting.

00:16:56.440 When I'm out and about in the world, I usually have to make the case for utilitarianism,

00:17:01.260 or at least you should consider this perspective.

00:17:03.320 Like people are scope insensitive.

00:17:05.620 You should look at the numbers.

00:17:07.440 If this thing has millions of people and this one only has hundreds of people being affected,

00:17:12.420 clearly that...

00:17:13.720 And yet when I'm back here at the headquarters, as it were, I usually am the one who has to

00:17:19.440 kind of advocate against the utilitarian perspective because so many of my friends are so deeply

00:17:24.420 dyed in the wool utilitarians.

00:17:26.740 And so narrowly focused on X-risk mitigation, but I feel that I'm always the odd one out.

00:17:34.220 Well, you know, I would love to get into a conversation with you about metaethics some

00:17:38.040 other time because I think your views about the limits of consequentialism would be fascinating

00:17:44.840 to explore.

00:17:45.880 But I have so much I want to talk to you about with respect to X-risk and a few of your papers

00:17:51.860 that I think, well, let's just table that for another time.

00:17:55.660 In fact, I don't even think we're going to be able to cover your book, Superintelligence.

00:17:59.960 I mean, maybe if we have a little time at the end, we'll touch it.

00:18:02.100 But I should just want to say that this book was incredibly influential on many of us in

00:18:08.220 arguing the case for there being a potential existential risk with respect to the development

00:18:14.760 of artificial intelligence and artificial general intelligence in particular.

00:18:19.780 And so it's, you know, this is something, the reason why I wouldn't cover this with you

00:18:23.700 for the entirety of this conversation is I've had several conversations on my podcast that

00:18:30.980 have been deeply informed by your view.

00:18:32.940 I mean, I've had Stuart Russell on, I've had Eliezer Yudkowsky on.

00:18:36.280 And basically, every time I talk about AI, I consider what I say to be, you know, fairly

00:18:42.620 derivative of your book.

00:18:44.500 And I often remind people of that.

00:18:46.680 So my audience will be familiar with your views on AI, even if they're not familiar with

00:18:51.860 you.

00:18:52.260 So if we have time, we'll get back to that.

00:18:54.440 But what I really want to talk about are a few of your papers.

00:18:58.000 The first is the Vulnerable World Hypothesis.

00:19:00.980 Maybe I'll just name the papers here that I hope we'll cover.

00:19:03.840 The Vulnerable World, the second is, are you living in a computer simulation?

00:19:09.600 And the third is, where are they?

00:19:11.500 Which is your analysis of the Fermi problem, asking where is the rest of the intelligent

00:19:17.660 life in the galaxy?

00:19:20.100 Let's start with the Vulnerable World Hypothesis.

00:19:23.220 What do you mean by that phrase?

00:19:25.980 Well, the hypothesis is, roughly speaking, that there is some level of technological development

00:19:31.660 at which the world gets destroyed by default, as it were.

00:19:36.620 So then, what does it mean to get destroyed by default?

00:19:39.940 I define something I call the semi-anarchic default condition, which is a condition in

00:19:45.820 which there are a wide range of different actors with a wide range of different human

00:19:50.580 recognizable motives.

00:19:51.640 But then, more importantly, two conditions hold.

00:19:56.020 One is that there is no very reliable way of resolving global coordination problems.

00:20:01.020 And the other is that we don't have a very extremely reliable way of preventing individuals

00:20:06.560 from committing actions that are extremely strongly disapproved of by a great majority of

00:20:12.440 other people.

00:20:14.160 Maybe it's better to come at it through a metaphor.

00:20:17.240 Yeah, the urn.

00:20:18.480 The urn metaphor.

00:20:19.520 So you can kind of think of the history of technological discovery as the process of pulling balls out

00:20:30.140 of a giant urn, the urn of creativity.

00:20:33.120 And we reach in and we get an idea out.

00:20:36.220 And then we reach in and get another out.

00:20:38.180 And we've extracted throughout history a great many of these balls.

00:20:42.600 And the net effect of this has been hugely beneficial, I would say.

00:20:47.120 And this is why we now sit in our air-conditioned offices and struggle not to eat too much rather

00:20:55.420 than to try to get enough to eat in large parts of the world.

00:20:58.200 But what if in this ball, in this urn, there is a black ball in there somewhere?

00:21:05.140 Is there some possible technology that could be such that whichever civilization discovers it,

00:21:13.020 environment gets destroyed?

00:21:14.360 Just to add a little color here, Nick.

00:21:16.720 So in your paper, you refer to this as the urn of inventions.

00:21:20.180 And we have been, as you say, pulling balls out as quickly as we can get our hands on them.

00:21:26.620 And on some level, the scientific ethos is really just a matter of pulling balls out as fast as you can

00:21:33.320 and making sure that everybody knows about them.

00:21:35.360 We have this norm of transparency in science.

00:21:39.500 And we have pulled out, thus far, only white or gray balls.

00:21:44.700 And the white balls are the ones, or the technologies, or the memes, or the norms, 0.62

00:21:48.920 or the social institutions that just have good consequences.

00:21:52.740 And the gray ones are norms and memes and institutions and, in most cases,

00:22:00.120 technology that has mixed results or that can be used for good or for ill.

00:22:05.420 And, you know, nuclear energy is a classic case where we can power our cities with it,

00:22:11.440 but we also produce fantastic amounts of pollution that's difficult to deal with.

00:22:16.480 And in the worst case, we build weapons.

00:22:19.440 So I just want to give a little more context to this analogy.

00:22:22.080 Yeah, and I guess most technologies are, in some sense, double-edged, but maybe the positive predominate.

00:22:29.140 I think there might be some technologies that are mainly negative if you think of, I don't know, nerve gases or other tools.

00:22:36.320 But what we haven't so far done is extract a black ball, right?

00:22:41.360 One that is so harmful that it destroys the civilization that discovers it.

00:22:46.160 And what if there is such a black ball in the urn, though?

00:22:50.420 I mean, we can ask about how likely that is to be the case.

00:22:54.040 We can also look at what is our current strategy with respect to this possibility.

00:22:59.160 And it seems to me that currently our strategy with respect to the possibility that the urn might contain a black ball is simply to hope that it doesn't.

00:23:07.780 And so we keep extracting balls as fast as we can.

00:23:10.880 We have become quite good at that, but we have no ability to put balls back into the urn.

00:23:16.260 We cannot uninvent our inventions.

00:23:18.020 So the first part of this paper tries to identify what are the types of ways in which the world could be vulnerable,

00:23:29.620 the types of ways in which there could be some possible black ball technology that we might invent.

00:23:34.440 And the first and most obvious type of way the world could be vulnerable is if there is some technology that greatly empowers individuals to cause sufficiently large quantities of destruction.

00:23:48.020 We motivate this with a, or illustrate it by means of a historical counterfactual.

00:23:54.520 We, in the last century, discovered how to split the atom and release the energy that is contained within some of the energy that's contained within the nucleus.

00:24:06.400 And it turned out that this is quite difficult to do.

00:24:10.700 You need special materials.

00:24:12.060 You need plutonium or highly enriched uranium.

00:24:13.760 So really only states can do this kind of stuff to produce nuclear weapons.

00:24:19.500 But what if it had turned out that there had been an easier way to release the energy of the atom?

00:24:24.220 What if you could have made a nuclear bomb by baking sand in the microwave oven or something like that?

00:24:30.880 So then that might well have been the end of human civilization in that it's hard to see how you could have cities, let us say,

00:24:38.120 if anybody who wanted to could destroy millions of people.

00:24:43.400 So maybe we were just lucky.

00:24:45.040 Now we know, of course, that it is physically impossible to create an atomic detonation by baking sand in the microwave oven.

00:24:53.120 But before you actually did the relevant nuclear physics, how could you possibly have known how it would turn out?

00:24:57.860 Well, let's just spell out that because I want to conserve everyone's intuitions as we go on this harrowing ride to your terminus here.

00:25:07.420 Because the punchline of this paper is fairly startling when you get to what the remedies are.

00:25:13.800 So why is it that civilization could not endure the prospect of what you call easy nukes?

00:25:23.920 If it were that easy to create a Hiroshima-level blast or beyond, 0.90

00:25:30.160 why is it just a foregone conclusion that that would mean the end of cities and perhaps the end of most things we recognize?

00:25:39.100 I think foregone conclusion is maybe a little too strong.

00:25:41.760 It depends a little bit on the exact parameters we plug in.

00:25:45.780 And the intuition is that in a large enough population of people,

00:25:50.380 like amongst every population with millions of people,

00:25:53.980 there will always be a few people who, for whatever reason,

00:25:57.880 would like to kill a million people or more if they could. 0.96

00:26:01.820 Whether they are just crazy or evil or they have some weird ideological doctrine 1.00

00:26:08.600 or they're trying to extort other people or threaten other people.

00:26:13.520 That just humans are very diverse and in a large enough set of people that will,

00:26:18.280 for practically any desire, you can specify there will be somebody in there that has that.

00:26:23.320 So if each of those destructively inclined people would be able to cause a sufficient amount of destruction,

00:26:29.140 then everything would get destroyed.

00:26:31.120 Now, if one in mind is this actually playing out in history,

00:26:37.480 then to tell whether all of civilization really would get destroyed

00:26:41.800 or some horrible catastrophe short of that would happen instead,

00:26:45.100 would depend on various things.

00:26:46.480 Like just what kind of nuclear weapon?

00:26:48.760 Would it be like a small kind of Hiroshima type of thing or a thermonuclear bomb?

00:26:53.260 How easy would it be?

00:26:54.400 Could literally anybody do it like in five minutes?

00:26:56.640 Or would it take some engineer working for half a year?

00:27:01.320 And so depending on exactly what values you pick for those and some other variables,

00:27:06.640 you might get scenarios ranging from very bad to kind of existential catastrophe.

00:27:13.540 But the point is just to illustrate that there historically have been these technological transitions

00:27:20.560 where we have been lucky in that destructive capability we discovered were hard to wield.

00:27:29.040 And maybe a plausible way in which this kind of very highly destructive capability

00:27:34.960 could become easy to wield in the future would be through developments in biotechnology

00:27:39.700 that maybe makes it easy to create designer viruses and so forth

00:27:43.480 that don't require high amounts of energy or special difficult materials and so forth.

00:27:50.560 And there you might have an even stronger case.

00:27:52.340 Like so with a nuclear weapon, like one nuclear weapon can only destroy one city, right?

00:27:57.120 Where the viruses and stuff potentially can spread.

00:28:01.080 So yeah, and we should remind people that we're in an environment now

00:28:05.380 where people talk with some degree of flippancy about the prospect of every household

00:28:14.740 one day having something like a desktop printer that can print DNA sequences, right?

00:28:20.160 That everyone becomes their own bespoke molecular biologist and you can just print your own medicine at home

00:28:27.880 or your own genetic intervention at home and this stuff really is, you know, the recipe under those conditions,

00:28:34.940 the recipe to weaponize the 1918 flu could just be sent to you like a PDF.

00:28:41.360 It's not beyond the bounds of plausible sci-fi that we could be in a condition where it really would be

00:28:49.240 within the power of one nihilistic or, you know, otherwise ideological person

00:28:54.480 to destroy the lives of millions and even billions in the wrong case.

00:28:59.080 Yeah, or send us a PDF or you could just download it from the internet.

00:29:02.800 That's the full genomes of the number of highly virulent organisms are in the public domain

00:29:08.780 and just ready to download.

00:29:11.520 So yeah, so I mean, we could talk more about that.

00:29:14.860 I think that I would rather see a future where DNA synthesis was a service provided by a few

00:29:19.940 places in the world where it would be able, if the need arose, to exert some control, some

00:29:25.280 screening rather than something that every lab needs to have its own separate little machine.

00:29:30.540 Yeah.

00:29:30.740 So that's, these are examples of type one vulnerability, like where the problem really

00:29:36.380 arises from individuals becoming too empowered in their ability to create massive amounts of

00:29:42.760 harm.

00:29:43.960 Now, so there are other ways in which the world could be vulnerable that are slightly more

00:29:48.800 subtle, but I think also worth bearing in mind.

00:29:51.760 So these have to do more about the way that technological developments could change the

00:29:59.200 incentives that different actors face.

00:30:01.200 We can again return to the nuclear history case for an illustration of this.

00:30:07.720 And actually, this is maybe the closest to a black ball we've gotten so far with thermonuclear

00:30:13.620 weapons and the big arms race during the Cold War led to something like 70,000 warheads

00:30:21.220 being on hair trigger alert.

00:30:23.320 So it looks loud.

00:30:24.720 Like when we can see some of the archives of this history that have recently opened up,

00:30:30.080 that there were a number of close calls.

00:30:32.860 The world actually came quite close to the brink on several occasions.

00:30:36.200 And we might have been quite lucky to get through.

00:30:38.680 It might not have been that we were in such a stable situation.

00:30:42.660 It rather might have been that this was a kind of slightly black ball-ish technology and

00:30:48.140 we just had enough luck to get through.

00:30:49.640 But you could imagine it could have been worse.

00:30:52.560 You could imagine properties of this technology that would have created stronger incentives,

00:30:56.280 say, for a first strike so that you would have crisis instability.

00:31:01.420 If it had been easier, let us say, in a first strike to take out all the adversary's

00:31:05.600 nuclear weapons, then it might not have taken a lot in a crisis situation to just have

00:31:14.300 enough fear that you would have to strike first or fear that the adversary otherwise would

00:31:19.160 do the same to you.

00:31:20.860 Yeah.

00:31:20.980 Remind people that in the aftermath of the Cuban Missile Crisis, the people who were closest

00:31:26.000 to the action felt that the odds of an exchange had been something like a coin toss.

00:31:31.620 It was something like 30 to 50 percent.

00:31:33.780 And what you're envisioning is a situation where what you describe as safe first strike, which

00:31:39.900 is, you know, there's just no reasonable fear that you're not going to be able to annihilate

00:31:44.540 your enemy provided you strike first.

00:31:46.820 That would be a far less stable situation.

00:31:49.600 And it's also forgotten that the status quo of mutually assured destruction was actually

00:31:56.020 a step towards stability.

00:31:57.920 I mean, there was before the Russians had or the Soviets had their own arsenals, there

00:32:03.940 was a greater, you know, game theoretic concern that we would be more tempted to use ours because

00:32:10.820 nuclear deterrence wasn't a thing yet.

00:32:14.240 Yeah.

00:32:14.680 So some degree of stabilizing influence, although, of course, maybe at the expense of, you know,

00:32:19.180 making it worse if you're also...

00:32:21.100 If you'd like to continue listening to this conversation, you'll need to subscribe at

00:32:26.540 samharris.org.

00:32:27.920 Once you do, you'll get access to all full-length episodes of the Making Sense podcast, along with

00:32:32.600 other subscriber-only content, including bonus episodes and AMAs and the conversations I've

00:32:38.180 been having on the Waking Up app.

00:32:40.080 The Making Sense podcast is ad-free and relies entirely on listener support.

00:32:44.680 And you can subscribe now at samharris.org.

Making Sense - Sam Harris - March 18, 2019

#151 — Will We Destroy the Future？

Episode Stats

Summary

Transcript