Making Sense of Artificial Intelligence ｜ Episode 1 of The Essential Sam Harris
Episode Stats
Length
1 hour and 7 minutes
Words per Minute
164.15674
Summary
In this episode of The Making Sense Podcast, host Sam Harris sits down with filmmaker Jay Shapiro to discuss his life, career, and work as an atheist. They talk about Jay's path to becoming a filmmaker, how he became interested in secularism, and why he decided to take his passion for secularism and secularism in particular to a whole new level. They also discuss his new project, "The End of Faith," a project he's working on with Jaron and Jaron to make a documentary about the life and career of atheist icon Christopher Hitchens. It's a fascinating conversation, and one that I think many of you should listen to. If you're interested in becoming a supporter of the podcast, please consider becoming a patron or a supporter, and if you haven't already become a patron, you'll get access to the full "Making Sense" catalog as well as access to all of the episodes of the Making Sense podcast available on all major podcast directories, including the most popular podcatcher, the Audible.org account. We don't run ads on the podcast and therefore it's made possible entirely through the support of our subscribers, so if you enjoy what we're doing here, you're making a generous donation. I am here to support what we re doing here...please consider becoming one! Thank you! Sam Harris - Make Sense: A Podcast About Stuff I'm Working on a Podcast by Jay Shapiro - The End Of Faith by Jay Shapiro - The Final Word by Sam Harris & Jaron J. Harris - The Other Side of the Podcast by Jaron jayshapoe Sam and J.J. Sam talks about how he got into atheistism, secularism & secularism by becoming a skeptic, and how to be a better atheist, and what it means to be an atheist by being an atheist in the 21st century by being a secularist by working to make sense of the world by working on the hard work of secularism and how he's going to do it better than most of us do it by being kinder, not better than we can do it Jay talks about what it looks like to be better than us all by being more than we know how to do what we can be, not more than that, and more like that, not less than we do it like that by being better than that And much more! and so much more, and much more.
Transcript
00:00:10.880
Just a note to say that if you're hearing this, you are not currently on our subscriber
00:00:14.680
feed and will only be hearing the first part of this conversation.
00:00:18.420
In order to access full episodes of the Making Sense Podcast, you'll need to subscribe at
00:00:24.060
There you'll find our private RSS feed to add to your favorite podcatcher, along with
00:00:30.260
We don't run ads on the podcast, and therefore it's made possible entirely through the support
00:00:35.880
So if you enjoy what we're doing here, please consider becoming one.
00:00:54.640
And let's see if I can remember the genesis of this.
00:00:59.200
I think, you know, I woke up in the middle of the night one night realizing that more
00:01:03.900
or less my entire catalog of podcasts was, if not the entire thing, maybe, you know, conservatively
00:01:11.360
speaking, you know, 50% of all the podcasts were evergreen, which is to say that their content
00:01:17.760
was basically as good today as the day I recorded them.
00:01:21.860
But because of the nature of the medium, they would never be perceived as such, and people
00:01:26.580
really don't tend to go back into the catalog and listen to, you know, a three-year-old podcast.
00:01:32.580
And yet there's something insufficient about just recirculating them in my podcast feed or
00:01:40.340
And so I and Jaron, my partner in crime here, we're trying to think about how to give all
00:01:51.160
And then we thought of you just independently turning your creative intelligence loose on
00:02:00.380
And now I will properly introduce you as someone who should be doing that.
00:02:07.480
Just tell us what you have done a lot of these many years and the kinds of things you've
00:02:13.720
Yeah, well, I'm a filmmaker first and foremost, but I think my story and my genesis of being
00:02:21.420
maybe the right person to tap here is probably indicative or representative of a decent portion
00:02:29.600
I'm 40 now, which pegs me in college when 9-11 hit.
00:02:37.400
I guess it would have been early if it was September.
00:02:39.640
And, you know, I never heard of you at all at that point.
00:02:43.620
I was an atheist and just didn't think too much about that kind of stuff.
00:02:47.820
I was fully on board with any atheist things I saw coming across my world.
00:02:52.840
But then 9-11 hit and I was on a very, very liberal college campus and the kind of questions
00:02:58.560
that were popping up in my mind and I was asking myself were uncomfortable for me.
00:03:05.840
I really had no formal philosophical training and I kind of just buried them, you know, under
00:03:11.340
under the weight of my own confusion or shame or just whatever kind of brew.
00:03:18.120
And then I discovered your work with The End of Faith, right when you sort of were responding
00:03:25.140
And a lot of your language, you were philosophically trained and maybe sharper with your language
00:03:31.920
for better or worse, which we found out later was complicated, resonated with me.
00:03:37.540
And I started following along with your work and The Four Horsemen and Hitchens and Dawkins
00:03:45.960
And then I paid close, special attention to what you were doing, which I actually included
00:03:52.720
in one of the pieces that I ended up putting together in this series.
00:03:56.860
But with a talk you gave in Australia, you know, I don't have to tell you about your career,
00:04:01.280
but again, I was following along as you were on sort of this atheist circuit and I was interested.
00:04:07.240
But whenever you would talk about sort of the hard work of secularism and the hard work
00:04:12.860
of atheism, this in particular, I'm thinking of your talk called Death in the Present Moment
00:04:19.780
I'm actually curious how quickly you threw that together because I know you were supposed
00:04:23.700
to or you were planning on speaking about free will and you ended up giving this whole
00:04:29.240
And that one, and I'll save it because I definitely put that one in our compilation.
00:04:32.840
But it struck me as, okay, this guy's up to something a little different and the questions
00:04:40.640
So I became a fan and like probably many of your listeners started to really follow and
00:04:49.560
And hopefully like any good student started to disagree with my teacher a bit and slowly
00:04:54.580
get the confidence to push back and have my own thoughts and maybe find the weaknesses
00:05:03.440
And, you know, your work exposed me and many, many other people, I'm sure, to a lot of great
00:05:08.480
And maybe you don't love this, but sometimes the people who disagree with you that you introduce
00:05:13.440
us to on this side of the microphone, we think are right.
00:05:19.960
And that's a great credit to you as well for just giving them the air and maybe on some
00:05:28.420
Because to back up way to the beginning of the story, I was at a university where I was
00:05:32.800
well on my way to a film degree, which is what I ended up getting.
00:05:35.420
But when 9-11 hit, I started taking a lot more courses in a track that they had, which
00:05:43.200
Maybe one, maybe still one of the only programs where you can actually major in Holocaust studies,
00:05:48.580
which is sort of sits in between the history and philosophy kind of departments.
00:05:54.380
And I started taking a bunch of courses in there.
00:05:57.420
And that's where I was first exposed to sort of the formal philosophy, language, and education.
00:06:06.020
And now hopefully I, you know, I swim deep in those waters and know my way around the
00:06:12.260
But yeah, it was almost, you know, Moore's law of bringing up the Nazis was those were the
00:06:17.600
first times actually in courses called like resistance during the Holocaust and things
00:06:22.380
like that, where, you know, I first was exposed to the words like deontology and consequentialism
00:06:27.660
and utilitarianism and a lot of moral ethics stuff.
00:06:30.260
And then I went further on my own into sort of the theory of mind and this kind of stuff.
00:06:34.980
But yeah, I consider myself in this weird new digital landscape that we're in a bit of a
00:06:41.780
But then again, like hopefully any good student, I've branched off and have my own sort of thoughts
00:06:47.660
And so that it's, I'm definitely in these pieces in this series of that we're calling
00:06:53.120
It is, I can't help but sort of put my writing and my framework on it, or at least hope that
00:06:59.600
the people and the challenges that you've encountered and continue to encounter, whether
00:07:04.920
they're right or wrong or making drastic mistakes, I want to give everything in it a really fair
00:07:11.500
So there's times I'm sure where the listener will hear my own hand of opinion coming in
00:07:17.080
there, and I'm sure you know the areas as well.
00:07:19.100
But most times I'm just trying to give an open door to the mystery and why these subjects
00:07:25.620
interest you in the first place, if that makes sense.
00:07:29.560
And I should remind both of us that we met because you were directing a film focused on
00:07:37.160
Majid Nawaz and me around our book, Islam and the Future of Tolerance.
00:07:42.500
And also we've brought into this project another person who I think you met independently, I
00:07:50.020
kind of remember, but Megan Phelps Roper, who's been a guest on the podcast and someone who I
00:07:55.920
have long admired, and she's doing the voiceover work in this series, and she happens to have
00:08:01.900
a great voice, so I'm very happy to be working with her.
00:08:06.820
Your archive, I think you said three or four years old, your archive is over 10 years old
00:08:11.500
And I was diving into the earliest days of it, and there are some fascinating conversations
00:08:20.540
And I'm curious, I mean, I think this project, again, it's for fans, it's for listeners, but
00:08:25.180
it's for people who might hate you also, or critics of you, or people who are sure you were
00:08:28.960
missing something or wrong about something, or even yourself, to go back and listen to
00:08:35.600
For example, one with like Dan Carlin, who hosts Hardcore History, you had him on, I think
00:08:39.660
that conversation is seven or eight years ago now.
00:08:42.220
And the part that I really resurfaced, it's actually in the morality episode, is full of
00:08:48.180
of details and philosophies and politics and moral philosophies regarding things like intervention
00:08:57.800
And at the time of your recording, of course, we had no idea how Afghanistan might look a
00:09:04.600
But now we kind of do, and it's not a, if people listen to these carefully, it's not
00:09:12.460
about, oh, this side of the conversation turned out to be right, and this kind of part turned
00:09:18.280
But certain things hit our ears a little differently.
00:09:21.800
Even on this first topic of artificial intelligence, I mean, I think that conversation continues
00:09:27.220
to be, evolve in a way where the issues that you bring up are evergreen, but hopefully evolving
00:09:35.220
as well, just as far as their application goes.
00:09:38.060
So yeah, so I think you, I would love to hear your thoughts listening back to some of those.
00:09:42.200
And in fact, to reference the film we made together, a lot of that film was you doing
00:09:46.480
that actively and live, given a specific topic of looking back and reassessing language
00:09:52.160
about how it might, you know, land politically in that project.
00:09:56.660
So yeah, but this, this goes into, to really different, including an episode about social
00:10:01.520
media, which changes every day, but fascinating to, yeah.
00:10:06.600
And the conversation you have with Jack Dorsey is now fascinating for all kinds of different
00:10:13.260
So yeah, it's evergreen, but it's also just like new life in all of them, I think.
00:10:20.640
Just to be clear, this has been very much your project.
00:10:24.960
I mean, I haven't heard most of this material since the time I recorded it and released
00:10:32.560
And, you know, you've gone back and created episodes on a theme where you've pulled
00:10:37.960
together five or six conversations and kind of intercut material from five or six different
00:10:45.380
episodes and then added your own interstitial pieces, which you have written and Megan Phelps
00:10:53.480
So it's just, these are very much, you know, their own documents.
00:10:57.360
And as you say, you don't agree with me about everything and you're occasionally you're, you're
00:11:02.000
shading different points from your own point of view.
00:11:04.840
And so, yeah, I look forward to hearing it and we'll be dropping the whole series here
00:11:13.000
If you're in the public feed, as always, you'll be getting partial episodes.
00:11:17.980
And if you're in the subscriber feed, you'll be getting full episodes.
00:11:22.480
And the first will be on artificial intelligence.
00:11:25.960
And then there are many other topics, consciousness, violence, belief, free will, morality, death,
00:11:34.780
There's one existential threat in nuclear war that I'm still piecing together, but it's,
00:11:47.880
I'm, again, I'm, I'm a consumer of this, probably more than a collaborator at this
00:11:53.440
point, because I have only heard part of what you've done here.
00:11:57.680
So I will be, I'll be eager to listen as well, but thank you for the work that you've
00:12:03.540
And I'll just say like, it's, it's, you, you're gracious to allow someone to do this
00:12:09.080
who, who does have some, you know, again, most of our, my disagreements with you are
00:12:13.900
pretty deep and nerdy and, and esoteric kind of philosophy stuff, but it's incredibly gracious
00:12:21.380
And then hopefully, again, I'm a bit of a representative for people who have been in
00:12:25.880
the passenger seat of your public project of thinking out loud for over a decade now.
00:12:30.920
And if I can, if I can, you know, be, be a voice for that, that part of the crowd, it's
00:12:39.340
And, and there are a lot of fun to a ton of fun.
00:12:41.120
There's a ton of audio, you know, like thought experiments that we play with and hopefully
00:12:44.720
bring to life in your ears a little bit, including in this very first one with artificial
00:12:52.900
So now we bring you Megan Phelps Roper on the topic of artificial intelligence.
00:13:03.100
This is making sense of artificial intelligence.
00:13:07.520
The goal of this series is to organize, compile, and juxtapose conversations hosted by Sam Harris
00:13:16.780
This is an ongoing effort to construct a coherent overview of Sam's perspectives and arguments,
00:13:23.020
the various explorations and approaches to the topic, the relevant agreements and disagreements,
00:13:28.580
and the pushbacks and evolving thoughts which his guests have advanced.
00:13:34.080
The purpose of these compilations is not to provide a complete picture of any issue, but
00:13:39.540
to entice you to go deeper into these subjects.
00:13:42.780
Along the way, we'll point you to the full episodes with each featured guest.
00:13:46.640
And at the conclusion, we'll offer some reading, listening, and watching suggestions, which range
00:14:00.080
Sam has long argued for a unity of knowledge where the barriers between fields of study are
00:14:05.080
viewed as largely unhelpful artifacts of unnecessarily partitioned thought.
00:14:09.080
The pursuit of wisdom and reason in one area of study naturally bleeds into, and greatly affects,
00:14:17.400
You'll hear plenty of crossover into other topics as these dives into the archives unfold.
00:14:22.960
And your thinking about a particular topic may shift as you realize its contingent relationships
00:14:29.320
In this topic, you'll hear the natural overlap with theories of identity and the self, consciousness,
00:14:48.680
Artificial intelligence is an area of resurgent interest in the general public.
00:14:53.680
Its seemingly eminent arrival first garnered wide attention in the late 60s, with thinkers
00:14:58.220
like Marvin Minsky and Isaac Asimov writing provocative and thoughtful books about the burgeoning technology
00:15:03.840
and concomitant philosophical and ethical quandaries.
00:15:06.580
Science fiction novels, comic books, and TV shows were flooded with stories of killer robots and encounters
00:15:13.680
with super-intelligent artificial lifeforms hiding out on nearby planets, which we thought
00:15:18.900
we would soon be visiting on the backs of our new rocket ships.
00:15:21.500
Over the following decades, the excitement and fervor look to have faded from view in the public imagination.
00:15:28.600
But in recent years, it has made an aggressive comeback.
00:15:32.300
Perhaps this is because the fruits of the AI revolution and the devices and programs once only imagined in those science fiction stories
00:15:39.360
have started to rapidly show up in impressive and sometimes disturbing ways all around us.
00:15:44.800
Our smartphones, cars, doorbells, watches, games, thermostats, vacuum cleaners, light bulbs, and glasses now have embedded algorithms
00:15:56.220
running on increasingly powerful hardware which navigate, dictate, or influence not just our locomotion,
00:16:03.180
but our entertainment choices, our banking, our politics, our dating lives, and just about everything else.
00:16:09.900
It seems every other TV show or movie that appears on a streaming service
00:16:14.700
is birthed out of a collective interest, fear, or otherwise general fascination
00:16:19.740
with the ethical, societal, and philosophical implications of artificial intelligence.
00:16:24.900
There are two major ways to think about the threat of what is generally called AI.
00:16:30.720
One is to think about how it will disrupt our psychological states or fracture our information landscape.
00:16:36.240
And the other is to ponder how the very nature of the technical details of its development may threaten our existence.
00:16:44.000
This compilation is mostly focused on the latter concern.
00:16:47.740
Because Sam is certainly amongst those who are quite worried about the existential threat
00:16:52.080
of the technical development and arrival of AI.
00:16:57.420
Now, before we jump into the clips, there are a few concepts that you'll need to onboard to find your footing.
00:17:02.760
You'll hear the terms Artificial General Intelligence, or AGI,
00:17:08.280
and Artificial Superintelligence, or ASI, used in these conversations.
00:17:13.980
Both of these terms refer to an entity which has a kind of intelligence
00:17:17.620
that can solve a nearly infinitely wide range of problems.
00:17:22.260
We humans have brains which display this kind of adaptable intelligence.
00:17:25.720
We can climb a ladder by controlling our legs and arms in order to retrieve a specific object
00:17:33.480
And we use the same brain to do something very different,
00:17:36.820
like recognize emotions in the tone of a voice of a romantic partner.
00:17:42.140
That same brain can play a game of checkers against a young child,
00:17:48.940
Or play a serious game of competitive chess against a skilled adult.
00:17:52.160
That same brain can also simply lift a coffee mug to our lips,
00:17:57.280
not just to ingest nutrients and savor the taste of the beans,
00:18:00.540
but also to send a subtle social signal to a friend at the table
00:18:03.980
to let them know that their story is dragging on a bit.
00:18:07.820
All of that kind of intelligence is embodied and contained in the same system,
00:18:17.800
which doesn't surpass what our brightest humans can accomplish on any given task,
00:18:22.220
while ASI references an intelligence which performs at,
00:18:29.300
This description of flexible intelligence is different from a system
00:18:32.680
which is programmed or trained to do one particular thing incredibly well,
00:18:38.760
or painting straight lines on the sides of a car,
00:18:48.340
based on the observable lifestyle habits of like-minded users
00:18:55.940
that is sometimes referred to as narrow or weak AI.
00:19:00.600
But even that kind of thing can be quite worrisome
00:19:03.100
from the standpoint of weaponization or preference manipulation.
00:19:07.220
You'll hear Sam voice his concerns throughout these conversations,
00:19:10.420
and he'll consistently point to our underestimation of the challenge
00:19:17.200
So, there are dangers and serious questions to consider
00:19:26.700
not everyone is as concerned about the technical existential threat of AI as Sam is.
00:19:34.680
stems from initial differences on the fundamental conceptual approach
00:19:41.860
Defining intelligence is notoriously slippery and controversial,
00:19:54.860
or an ability to manifest preferred future states
00:19:58.000
through intentional current action and intervention.
00:20:00.560
You can imagine a linear gradient indicating more or less
00:20:05.140
of the amount of this competence as you move along it.
00:20:09.320
This view places our human intelligence on a continuum
00:20:12.360
along with bacteria, ants, chickens, honeybees, chimpanzees,
00:20:18.280
all of the potential undiscovered alien lifeforms,
00:20:24.480
which perches itself far above our lowly human competence.
00:20:34.940
that perhaps we shouldn't be actively seeking out
00:20:41.500
which is far more technologically advanced than ours.
00:20:44.820
And, if our planet's history provides any lessons,
00:20:48.020
it seems to prove that when technologically mismatched cultures
00:20:52.340
it usually doesn't work out too well for the lesser-developed one.
00:20:55.460
Are we bringing that precise suicidal encounter into reality
00:20:59.920
as we set out to develop artificial intelligence?
00:21:03.320
That question alludes to what is known as the value alignment problem.
00:21:11.400
which starts to lay out the important definitional foundations
00:21:14.100
and distinction of terms in the landscape of AI.
00:21:19.440
is the decision theorist and computer scientist Eliezer Yudkowsky.
00:21:23.140
Yudkowsky begins here by defending this linear gradient perspective on intelligence
00:21:45.160
about the implications that all of this has for our future.
00:22:22.640
would give you the same definition of intelligence.
00:23:26.340
And we think it has something to do with our brains.
00:23:30.200
I think we can make it more abstract than that.
00:23:33.160
Tell me if you think this is not generic enough
00:23:37.880
Whatever intelligence may be in specific context,
00:23:47.660
perhaps across a diverse range of environments.
00:24:02.740
following the same strategy again and again blindly.
00:24:05.480
Does that seem like a reasonable starting point?
00:24:13.780
with some of the things that are in AI textbooks.
00:24:15.760
If I'm allowed to sort of take it a bit further
00:25:30.400
our competitor in terms of general optimization
00:48:16.440
you tell it to take you to the airport as fast as
00:48:22.760
like this is not what i asked for and it replies
00:48:25.760
that is exactly what you asked for then you realize how hard it is to get that
00:48:32.420
machine to learn your goals right if you tell an uber driver to take you to the
00:48:36.680
airport as fast as possible she's going to know that you actually had
00:48:40.200
additional goals that you didn't explicitly need to say because she's a
00:48:44.760
human too and she understands where you're coming from but for someone made
00:48:49.080
out of silicon you have to actually explicitly have it learn all of those
00:48:54.440
other things that we humans care about so that's hard and then once it can
00:48:59.500
understand your goals that doesn't mean it's going to adopt your goals i mean
00:49:05.640
and finally if you get the machine to adopt your goals then
00:49:11.200
how can you ensure that it's going to retain those goals and
00:49:14.680
gradually gets smarter and smarter through self-improvement
00:49:18.780
most of us grown-ups have pretty different goals from what we had when we were
00:49:24.120
five i'm a lot less excited about legos now for example and uh we don't want a
00:49:30.800
super intelligent ai to just think about this goal of being nice to humans and some
00:49:35.580
some little passing fad from its early youth it seems to me that the second
00:49:41.980
scenario of value alignment does imply the first of keeping the ai successfully
00:49:48.540
boxed at least for a time because you have to be sure its value aligned before
00:49:54.980
you let it out in the world before you let it out on the internet for instance or
00:49:59.640
you create you know robots that have superhuman intelligence that are
00:50:04.480
functioning autonomously out in the world do you see a development path where we
00:50:09.820
don't actually have to solve the the boxing problem at least initially no i think
00:50:16.340
you're completely right even if your intent is to build a value line ai and let it out
00:50:20.480
you clearly are going to need to have it boxed up during the development phase when you're
00:50:24.620
just messing around with it just like any biolab that deals with dangerous pathogens is very
00:50:31.140
carefully sealed off and uh it's this highlights the incredibly pathetic state of computer
00:50:38.560
security today i mean and i think pretty much everybody who listens to this has at some
00:50:43.140
point experienced the blue screen of death courtesy of microsoft windows or the spinning wheel of doom
00:50:48.800
courtesy of apple and we need to get away from that to have truly robust machines if we're ever going
00:50:56.780
to be able to have ai systems that we can trust that are provably secure and i feel it's actually
00:51:03.860
quite embarrassing that we're so flippant about this it's it's maybe annoying if your computer
00:51:10.600
crashes and you lose one hour of work that you hadn't saved but it's not as funny anymore if it's
00:51:16.180
yourself driving car that crashed or the control system for your nuclear power plant or your nuclear
00:51:21.960
weapon system or something like that and when we start talking about human level ai and boxing systems
00:51:28.340
you have to have this much higher level of safety mentality where you've really made this a priority
00:51:35.080
the way we aren't doing today yeah you describe in the book various catastrophes that have happened
00:51:41.880
by virtue of software glitches or just bad user interface where you know the dot on the screen or
00:51:48.360
the number on the screen is is too small for the human user to deal with in real time and so we there
00:51:54.420
have been plane crashes where scores of people have died and patients have been annihilated by having you
00:52:01.940
know hundreds of times the radiation dose that they should have gotten in various machines because
00:52:07.900
the the software was improperly calibrated or the user had selected the wrong option and so we're by no
00:52:14.580
means perfect at this even when we have a human in the loop and here we're talking about systems that
00:52:23.440
we're creating that that are going to be fundamentally autonomous and you know the idea of having
00:52:29.660
perfect software that has been perfectly debugged before it assumes these massive responsibilities
00:52:37.260
it is fairly daunting i mean just i mean how do we recover from something like you know seeing the
00:52:43.840
stock market go to zero because we didn't understand the ai that we we unleashed on the on you know the dow
00:52:52.020
jones or the financial system generally i mean these are these are not impossible outcomes
00:52:58.600
yeah you you raise a very important point there just to inject some optimism in this i do want to
00:53:05.060
emphasize that first of all there's a huge upside also if one can get this right because people are
00:53:11.400
bad at things yeah in all of these areas where there were horrible accidents of course the technology can
00:53:15.820
save lives and health care and transportation and so many other areas so there's an incentive to do it
00:53:21.340
and secondly there are examples in history where we've had really good safety engineering
00:53:27.260
built in from the beginning for example when we sent neil armstrong buzz aldrin and michael collins to
00:53:33.720
they did not die there were tons of things that could have gone wrong but nasa very meticulously
00:53:39.880
tried to predict everything that possibly could go wrong and then take precautions so it didn't happen
00:53:45.920
right they weren't luck it wasn't luck that got them there it was planning and i think we need to shift
00:53:51.480
into this safety engineering mentality with ai development throughout history it's always been
00:53:58.880
the situation that we could we could create a better future with technology as long as we
00:54:03.020
won this race between the growing power of the technology and the growing wisdom with which we
00:54:08.600
managed it and in the past we by and large used the strategy of learning from mistakes to stay ahead in
00:54:15.600
the race we invented fire oopsie screwed up a bunch of times and then we uh invented the fire extinguisher
00:54:21.480
we uh invented cars oopsie and invented the seat belt but with more powerful technology like
00:54:28.260
nuclear weapons synthetic biology super intelligence we don't want to learn from mistakes that's a
00:54:36.080
terrible strategy we instead want to have a safety engineering mentality where we plan ahead and
00:54:42.760
get things right the first time because that might be the only time we have
00:54:46.780
it's helpful to note the optimism that tegmark plants in between the flashing warning signs
00:54:53.840
artificial intelligence holds incredible potential to bring about inarguably positive changes for
00:55:00.820
humanity like prolonging lives eliminating diseases avoiding all automobile accidents increasing logistic
00:55:09.220
efficiency in order to deliver food or medical supplies cleaning the climate increasing crop yields
00:55:15.480
expanding our cognitive abilities to learn languages or improve our memory the list goes on imagine being
00:55:23.060
able to simulate the outcome of a policy decision with a high degree of confidence in order to morally
00:55:28.160
assess it consequentially before it is actualized now some of those pipe dreams may run contrary to the
00:55:35.020
laws of physics but the likely possible positive outcomes are so tempting and morally compelling that the
00:55:41.160
urgency to think through the dangers is even more pressing than it first seems
00:55:45.020
tegmark's book on the subject where much of that came from is fantastic it's called life 3.0 just a
00:55:53.140
reminder that a reading watching and listening list will be provided at the end of this compilation
00:55:57.240
which will have all the relevant texts and links from the guests featured here somewhere in the middle of
00:56:03.440
the chronology of these conversations sam delivered a ted talk that focused on and tried to draw attention to
00:56:08.900
the value alignment problem much of his thinking about this entire topic was heavily influenced by the
00:56:15.260
philosopher nick bostrom's book superintelligence sam had nick on the podcast though their conversation
00:56:21.740
delved into slightly different areas of existential risk and ethics which belong in other compilations
00:56:26.900
but while we're on the topic of the safety and promise of ai we'll borrow some of bostrom's helpful
00:56:33.380
frameworks bostrom draws up a taxonomy of four paths of development for an ai each with its own safety and
00:56:43.500
control conundrums he calls these different paths oracles genies sovereigns and tools an artificially
00:56:53.620
intelligent oracle would be a sort of question and answer machine which we would simply seek advice from
00:56:58.940
it wouldn't have the power to execute or implement its solutions directly that would be our job think
00:57:05.760
of a super intelligent wise sage sitting on a mountaintop answering our questions about how to
00:57:10.900
solve climate change or cure a disease the ai genie and an ai sovereign both would take on a wish or
00:57:19.560
desired outcome which we impart to it and pursue it with some autonomy and power to achieve it out in the
00:57:25.320
world perhaps it would work in concert with nanorobots or some other networked physical
00:57:30.860
entities to do its work the genie would be given specific wishes to fulfill while the sovereign might
00:57:36.960
be given broad open-ended long-range mandates like increase flourishing or reduce hunger and lastly the
00:57:45.620
tool ai would simply do exactly what we command it to do and only assist us to achieve things we already
00:57:51.560
knew how to accomplish the tool would forever remain under our control while completing our tasks and
00:57:58.000
easing our burden of work there are debates and concerns about the impossibility of each of these
00:58:03.360
entities and ethical concerns about the potential consciousness and immoral exploitation of any of
00:58:09.060
these inventions but we'll table those notions just for a bit this next section digs in deeper on the
00:58:16.540
ideas of a genie or a sovereign ai which is given the ability to execute our wishes and commands
00:58:22.400
autonomously can we be assured that the genie or sovereign will understand us and that its values
00:58:28.620
will align in crucial ways with ours in this clip stuart russell a professor of computer science at cal
00:58:36.640
berkeley gets us further into the value alignment problem and tries to imagine all the possible ways that
00:58:42.940
having a genie or sovereign in front of us might go terribly wrong and of course what we might be
00:58:49.360
able to do to make it go phenomenally right sam considers this issue of value alignment central to
00:58:56.100
making any sense of ai so this is stuart russell from episode 53 the dawn of artificial intelligence
00:59:04.480
let's talk about that issue of what bostrom called the control problem i guess we call it the safety
00:59:13.720
problem just perhaps you can briefly sketch the concern here what is what is the concern about
00:59:20.100
general ai getting away from us how do you articulate that um so you mentioned earlier that this is a
00:59:28.820
concern that's being articulated by non-computer scientists and bostrom's book super intelligence
00:59:33.900
was certainly instrumental in bringing it to the attention of a of a wide audience you know people
00:59:39.520
like bill gates and elon musk and so on but the fact is that these concerns have been articulated by
00:59:47.040
the central figures in computer science and ai so i'm actually gonna going back to ij good and von
00:59:55.300
neumann uh well and and alan turing himself right um so people a lot of people may not know about this
01:00:04.680
i'm just gonna read a little quote so alan turing gave a talk on uh bbc radio radio three in 1951
01:00:15.760
um so he said if a machine can think it might think more intelligently than we do and then where should
01:00:23.920
we be even if we could keep the machines in a subservient position for instance by turning off
01:00:29.340
the power at strategic moments we should as a species feel greatly humbled this new danger is
01:00:36.200
certainly something which can give us anxiety so that's a pretty clear you know if we achieve
01:00:42.420
super intelligent ai we could have uh a serious problem another person who talked about this
01:00:49.220
issue was norbert wiener uh so norbert wiener was the uh one of the leading applied mathematicians
01:00:57.360
of the 20th century he was uh the founder of a good deal of modern control theory
01:01:03.900
um and uh automation site he's uh often called the father of cybernetics so he was he was concerned
01:01:13.120
because he saw arthur samuel's checker playing program uh in 1959 uh learning to play checkers
01:01:20.520
by itself a little bit like the dqn that i described learning to play video games but this is 1959 uh so
01:01:27.880
more than 50 years ago learning to play checkers better than its creator and he saw clearly in this
01:01:35.440
the seeds of the possibility of systems that could out distance human beings in general so and he he was
01:01:43.940
more specific about what the problem is so so turing's warning is in some sense the same concern that
01:01:49.780
gorillas might have had about humans if they had thought you know a few million years ago when the
01:01:55.740
human species branched off from from the evolutionary line of the gorillas if the gorillas had said to
01:02:01.120
themselves you know should we create these human beings right they're going to be much smarter than
01:02:04.640
us you know it kind of makes me worried right and and the probably they would have been right to worry
01:02:09.780
because as a species they're they sort of completely lost control over their own future and and humans
01:02:16.240
control everything that uh that they care about so so turing is really talking about this general sense of
01:02:24.340
unease about making something smarter than you is that a good idea and what wiener said was was this if
01:02:30.460
we use to achieve our purposes a mechanical agency with whose operation we cannot interfere effectively
01:02:37.280
we had better be quite sure that the purpose put into the machine is the purpose which we really desire
01:02:44.240
so this 1660 uh nowadays we call this the value alignment problem how do we make sure that the
01:02:52.420
the values that the machine is trying to optimize are in fact the values of the human who is trying to
01:03:00.560
get the machine to do something or the values of the human race in in general um and so we know
01:03:07.980
actually points to the sorcerer's apprentice story uh as a typical example of when when you give
01:03:15.900
uh a goal to a machine in this case fetch water if you don't specify it correctly if you don't cross
01:03:24.260
every t and dot every i and make sure you've covered everything then machines being optimizers they will
01:03:31.360
find ways to do things that you don't expect uh and those ways may make you very unhappy uh and this
01:03:38.980
story goes back you know to king midas uh you know 500 and whatever bc um where he got exactly what he
01:03:48.200
said which is the thing turns to gold uh which is definitely not what he wanted he didn't want his
01:03:54.060
food and water to turn to gold or his relatives to turn to gold but he got what he said he wanted
01:03:59.180
and all of the stories with the genies the same thing right you you give a wish to a genie the genie
01:04:04.880
carries out your wish very literally and then you know the third wish is always you know can you undo
01:04:09.600
the first two because i got them wrong and the problem with super intelligent ai uh is that you
01:04:16.360
might not be able to have that third wish or even a even a second wish yeah so if you so if you get it
01:04:22.500
wrong you might wish for something very benign sounding like you know could you cure cancer but if
01:04:27.880
if you haven't told the machine that you want cancer cured but you also want human beings to be
01:04:34.060
alive so a simple way to cure cancer in humans is not to have any humans um a quick way to come up
01:04:40.880
with a cure for cancer is to use the entire human race's guinea pigs or for millions of different
01:04:46.980
essential uh drugs that might cure cancer um so there's all kinds of ways things can go wrong and
01:04:53.080
you know we have you know governments all over the world try to write tax laws that don't have these
01:05:01.180
kinds of loopholes and they fail over and over and over again and they're only competing against
01:05:07.720
ordinary humans you know tax lawyers and rich people um and yet they still fail despite there being
01:05:16.680
billions of dollars at stake so our track record of being able to specify objectives and constraints
01:05:26.560
completely so that we are sure to be happy with the results our track record is is abysmal and
01:05:33.920
unfortunately we don't really have a scientific discipline for how to do this so generally we have
01:05:40.920
all these scientific disciplines ai control theory economics operations research that are about
01:05:49.020
how do you optimize an objective but none of them are about well what should the objective be so that
01:05:55.000
we're happy with the results so that's really i think the modern understanding uh as described
01:06:03.440
in bostrom's book and other papers of why a super intelligent machine could be problematic it's
01:06:10.120
because if we give it an objective which is different from what we really want then we we're basically
01:06:17.540
like creating a chess match with a machine right now there's us with our objective and it with
01:06:22.720
the objective we gave it which is different from what we really want so it's kind of like having
01:06:27.460
a chess match for the whole world uh and we're not too good at beating machines at chess
01:06:33.540
throughout these clips we've spoken about ai development in the abstract as a sort of
01:06:41.280
technical achievement that you can imagine happening in a generic lab somewhere but this next clip is going
01:06:47.320
to take an important step and put this thought experiment into the real world if this lab does create
01:06:55.040
something that crosses the agi threshold the lab will exist in a country and that country will have
01:07:01.020
alliances enemies paranoias prejudices histories corruptions and financial incentives like any country
01:07:09.400
how might this play out if you'd like to continue listening to this conversation you'll need to
01:07:17.060
subscribe at samharris.org once you do you'll get access to all full-length episodes of the making
01:07:22.180
sense podcast along with other subscriber only content including bonus episodes and amas and the
01:07:28.680
conversations i've been having on the waking up app the making sense podcast is ad free and relies
01:07:33.980
entirely on listener support and you can subscribe now at samharris.org