Making Sense - Sam Harris - #332 — Can We Contain Artificial Intelligence？

00:00:00.000 welcome to the making sense podcast this is sam harris just a note to say that if you're hearing

00:00:12.500 this you are not currently on our subscriber feed and will only be hearing the first part

00:00:16.900 of this conversation in order to access full episodes of the making sense podcast you'll

00:00:21.800 need to subscribe at sam harris.org there you'll find our private rss feed to add to your favorite

00:00:27.020 podcatcher along with other subscriber only content we don't run ads on the podcast and

00:00:32.500 therefore it's made possible entirely through the support of our subscribers so if you enjoy

00:00:36.540 what we're doing here please consider becoming one welcome to the making sense podcast this is sam

00:00:49.320 harris okay just a reminder that subscribers to the podcast can now share full episodes by going

00:00:58.600 to the episode page on my website and getting the link and you can share one-to-one with friends and

00:01:06.740 family or you can post to social media whatever you like okay today i'm speaking with mustafa

00:01:16.020 suleiman mustafa is the co-founder and ceo of inflection ai and a venture partner at graylock

00:01:23.520 a venture capital firm before that he co-founded deep mind which is one of the world's leading

00:01:29.980 artificial intelligence companies now part of google and he was vice president of ai product management

00:01:36.940 and ai policy at google and he is also the author of a new book the coming wave technology power and

00:01:45.880 the 21st century's greatest dilemma which is the focus of today's conversation we talk about the

00:01:51.820 new book we talk about the progress that was made in ai by his company deep mind various landmarks they

00:02:00.380 achieved atari dqn alpha go alpha zero alpha fold we discuss the amazing fact that we now have

00:02:08.820 technology that can invent new knowledge the risks of our making progress in ai super intelligence as a distraction

00:02:16.840 from more pressing problems the inevitable spread of general purpose technology the nature of intelligence

00:02:23.220 productivity growth and labor disruption the containment problem the importance of scale

00:02:29.920 open source llms changing norms of work and leisure the redistribution of value introducing friction into the deployment

00:02:39.680 of ai regulatory capture the looming possibility of a misinformation apocalypse digital watermarks asymmetric

00:02:49.060 threats conflict and cooperation with china supply chain monopolies and other topics anyway it was great to get

00:02:58.020 mustafa here he's one of the pioneers in this field and as you'll hear he shares many of my concerns

00:03:03.560 but with different points of emphasis and now i bring you mustafa suleiman

00:03:09.360 i am here with mustafa suleiman mustafa thanks for joining me great to be with you sam thanks for having me

00:03:20.640 so you you have a new book which um the world needs because this is the problem of our time the title is

00:03:28.420 the coming wave technology power and the 21st century's greatest dilemma

00:03:34.000 uh and we will we'll get into the book because um it's really um quite a good read and uh we will

00:03:42.200 talk about what that coming wave is but you're especially concerned about ai which is your um your

00:03:49.260 wheelhouse but also you're talking about synthetic biology uh and to i guess a lesser degree robotics

00:03:56.540 and some other technologies that are going to be more and more present if things don't run totally off

00:04:02.560 the rails for us but before we we jump into the book let's talk about your background but how would

00:04:08.060 you describe the bona fides that have have brought you to this conversation yeah i mean i i started life

00:04:15.740 i guess as a entrepreneur when i was 18 i started my first company which was a point of sale system uh sales

00:04:24.140 company and we we were sort of installing these these sort of very early pdas back in 2002 2003 and networking

00:04:33.340 equipment i wasn't successful uh but that was my first attempt i i dropped out of oxford uh the end of my

00:04:40.380 second year where i was reading philosophy to start a charity and i helped two or three other people

00:04:47.100 get a telephone counseling service off the ground it was a secular service for young british muslims i was

00:04:54.220 a just turned an atheist at the time having been to oxford discovered human rights principles and the

00:05:01.740 ideas of universal justice and uh managed to sort of move out of the faith and uh and decided that i i

00:05:10.220 i really wanted to you know dedicate my life to to doing good and and studying philosophy and the theory

00:05:16.700 uh was too too esoteric and and too distant from action i'm a very kind of practical action focused person

00:05:25.260 so i spent a couple years doing that a little bit after that time i spent a year or so working in

00:05:31.580 local government as a human rights policy officer for the mayor of london at the time i think i was

00:05:37.180 21 when i started that job it was very big and exciting but ultimately quite unsatisfying and

00:05:43.660 frustrating who was the mayor was that johnson that was before johnson yeah quite a bit before uh it was

00:05:50.140 it was ken livingston back in 2004 right so quite a while back in london and then from there i wanted to

00:05:58.860 see how i could scale up my impact in the world and um you know i i helped to start a conflict

00:06:06.780 resolution firm i was very lucky at the age of 22 to be able to co-found this this consultancy with

00:06:14.220 a group of some of the most practiced uh negotiation experts in the world some of the people who are

00:06:21.180 involved in the peace and reconciliation process in south africa post-apartheid and there's a big group

00:06:26.220 of us coming together with very different skills and backgrounds and uh i had incredible three years

00:06:31.580 there working all over the world in in cyprus and for the dutch government for you know on the israel

00:06:38.620 palestine question uh you know many different places and it was hugely inspiring and taught me

00:06:44.860 a lot about the world but i sort of fundamentally realized from there that if i if i didn't get back

00:06:50.220 to technology i would miss the most important transition you know wave if you like happening

00:06:55.420 in my lifetime and uh you know i i said about shortly after the climate negotiations that we were working

00:07:04.140 on in 2009 in copenhagen everyone left feeling frustrated and disappointed that we hadn't managed

00:07:11.020 to reach agreement and this was the year that sort of obama was coming over and everyone had a lot of hope

00:07:16.140 and it didn't happen uh it turns out for another 10 or 12 years and i i sort of had this aha moment i

00:07:22.860 was like if i don't get back to technology then i'm going to miss the most important thing happening and

00:07:27.820 so i set about on this quest trying to sort of find anyone that any you know anyone who i knew even

00:07:34.380 tangentially who was working in technology and uh my best friend from when we were teenagers um his older

00:07:42.620 brother was demis asabis and uh we we were playing poker together one night in the victoria casino in

00:07:49.500 london and and we got chatting about the ways that at the time you know we framed it as robots were

00:07:55.100 going to transform the world and deliver enormous productivity boosts and improve efficiency in every

00:08:01.100 respect and we were sort of debating like how do you do good in the world how do you get things done

00:08:06.780 you know what what is the real set of incentives and efforts that really makes a difference

00:08:12.220 and um you know both very passionate about science and technology and having a positive

00:08:16.620 impact in the world and you know one thing led to another and eventually we um ended up starting deep

00:08:22.540 mind and i did that for 10 years yeah along with shane leg right shane is uh is our other co-founder

00:08:31.580 exactly shane was a gatsby computational neuroscience unit in london at the time and he was just finishing

00:08:37.660 uh he had just finished his phd a few years earlier he was doing postdoctoral research

00:08:42.220 and his phd was on definitions of intelligence which was super interesting it was very obscure and

00:08:48.380 really really relevant he was sort of trying to synthesize 60 or so different definitions of

00:08:54.140 intelligence and trying tried to sort of abstract that into an algorithmic construct one that we could use to

00:09:00.300 measure progress towards some defined goal and um his frame was that intelligence is the ability to

00:09:07.900 perform well across a wide range of environments so the core emphasis was that intelligence was about

00:09:14.940 generality right and we you know we can get into this there's lots of different definitions of

00:09:19.340 intelligence which place emphasis on different aspects of our capabilities but generality has become

00:09:25.900 the core concept that sort of dominated the field for the last sort of 12 15 years and and of course

00:09:32.540 the term agi i mean that that predated shane but i think it was very much popularized by our kind of

00:09:40.220 mission you know sort of it was really the first time in a long time that a company had been founded to

00:09:47.420 invent general intelligence or agi and that was our mission to try and build safe and ethical artificial

00:09:54.140 general intelligence so i'm trying to remember where we met i know we were both at the puerto rico

00:09:59.900 conference at the beginning of 2015 that uh i think i don't know if it was the first of these meetings

00:10:07.900 but it was the first that i was aware of that really focused the conversation on ai safety and risk

00:10:13.820 and i know i met demis there i'm not sure i think you and i met in la subsequent to that is that right

00:10:20.460 yeah i think i think we we met i can't remember if we met before or after that but i think we had we

00:10:26.780 had a common interest in our la conversation it might have been just before that talking about

00:10:32.540 extremism and radicalization and terrorism and oh within islam yeah yeah that's right yeah yeah so yeah

00:10:39.340 i can't i don't think we met in puerto rico but that that conference was very formative of my

00:10:44.140 you know it was really like my first impression of of how big a deal this was going to be ultimately

00:10:50.220 and then there was a subsequent conference in 2017 at asilimar where i think we uh we met again and i

00:10:58.860 think i met shane there as well so let's before we jump into again the book and what you're doing

00:11:05.260 currently because you've you've since moved on from deep mind and you have a new company that we'll talk

00:11:09.740 about but um let's talk about deep mind because it really was you know before it was you know it's

00:11:17.980 been eclipsed in the popular consciousness by open ai of late with with the advent of chat gpt and

00:11:26.540 and uh large language models but prior to that really deep mind was the preeminent

00:11:33.820 uh it may in fact still be the preeminent ai company but it's now a branch of google give me

00:11:41.980 give us a little bit of the history there and and tell us what was accomplished because you at deep

00:11:49.020 mind you had several breakthroughs that were just fundamental and you really put ai back on the map and

00:11:57.740 prior to what you did there we were in an ai so-called ai winter where it was just common

00:12:04.460 knowledge that this artificial intelligence thing wasn't really panning out and then all of a sudden

00:12:09.420 everything changed so i think pre-acquisition um which was in 2014 i think our there were probably two

00:12:17.100 principal contributions that we made i think the first is we made a very early bet on deep learning i

00:12:24.140 mean the company was founded in 2010 in the summer of 2010 and it really wasn't for a couple of years

00:12:30.780 that deep learning had even appeared on the on the field even academically with the image net challenge

00:12:36.140 a few years after we founded so that was a very significant bet that we made early and that we got

00:12:41.660 right and the consequence of that was that we were able to hire some of the best phds and postdoctoral

00:12:48.780 researchers in the world you know who at the time were working on this very obscure very uninteresting

00:12:55.820 you know largely not very valuable subject in fact jeff hinton was one of our consultants so was his

00:13:04.300 student at the time ilia satskiva who's now chief scientist and co-founder of open ai along with many

00:13:10.860 others from open ai and elsewhere who you know basically either worked with us full-time or worked with

00:13:16.700 us as as consultants and that was largely you know reflective of the fact that we got the bet right

00:13:23.180 early on deep learning the second contribution i would say was the combination of deep learning and

00:13:29.500 reinforcement learning i mean if deep learning was was obscure reinforcement learning was even more

00:13:35.020 theoretical and you know we were actually quite careful to frame our mission among academics you know

00:13:42.620 less around sort of agi and more around applied machine learning you know there was a certainly

00:13:48.620 in the very early days we're a bit hush-hush about it but you know as we got more traction in 2011 2012

00:13:54.540 it became very attractive to people who were otherwise quite theoretical in their outlook to come and

00:14:00.620 work on problems like reinforcement learning in you know sort of more engineering focused setting albeit

00:14:06.300 still a research lab and it was the combination of deep learning and reinforcement learning that

00:14:11.820 led to our first i think major contribution which was the atari dqn ai dqn so you know dqn was a pretty

00:14:23.660 incredible system i mean it essentially learned to play 50 or so of the old school sort of 80s ataris

00:14:31.100 games atari games to human level performance uh simply from the pixels learning to correlate a set of

00:14:38.700 rewarding moments in the game via score with a set of frames that led to that score in the run-up to

00:14:46.380 that and the actions that were taken there and that was a really significant achievement it was actually

00:14:50.940 that which caught larry page's attention and led him to email us you know and and you know sort of invite

00:14:59.420 us to come and be part of of google and then google acquired you and uh what was the logic there you

00:15:07.340 just it was good to have google's resources to scale or i mean larry made a very simple claim which was

00:15:16.700 you know i've spent you know the last you know 10 years or so building a platform with all the resources

00:15:25.100 necessary to make a really big bet on agi you know why should you guys go through all of that again

00:15:32.140 you know we'll give you the freedom you need to carry on operating as a essentially a you know

00:15:37.500 independent subsidiary even though we were part of google why wouldn't you just come and work with us

00:15:42.780 and have all the resources you need to to scale you know significantly which is what we did and it's it's

00:15:49.020 it was a very compelling proposition because at the time you know monetizing deep learning back

00:15:54.940 in 2014 was going to be really tough so but google had its own ai division as well that was just kind

00:16:03.420 of working in parallel with deep mind did that did you guys at some point you guys merged i don't know

00:16:09.740 if that happened after you left or before but how did that was there a firewall between the two

00:16:16.060 divisions for a time and then that came down or how'd that work yeah so the division you're referring

00:16:22.380 to is google brain which is run by jeff dean and i think that started in 2015 with andrew ung actually

00:16:29.180 as well and you know in some ways that's the kind of beauty of google's scale right that it was able to

00:16:35.740 run multiple huge billion dollar efforts in parallel and the merger which i think has been long coming

00:16:43.100 actually only happened this year so google plus deep mind is now google deep mind and uh most of

00:16:50.540 the kind of open-ended research on ai is now consolidated around google deep mind and all of

00:16:56.780 the sort of more focused applied research that helps google products more directly in the short term is is

00:17:03.740 focused on a on a separate division google research right so you had those the the atari game

00:17:11.100 breakthrough which caught everyone's attention because you either you have these you know if

00:17:16.220 memory serves you managed to build a system that had i mean it achieved human level competence and

00:17:25.740 beyond and also achieved novel strategies that many humans wouldn't come up with but then the the

00:17:33.500 real breakthroughs that got everyone's attention were with alpha go and alpha zero and alpha fold perhaps

00:17:40.860 you can run through those because that that's when at least to my eye things just became unignorable

00:17:47.580 in the ai field yeah that that's exactly right i mean it it's it's it's pretty interesting because sort

00:17:54.060 of after we got acquired it was actually sergey that was sort of insisting that we tackle go i mean his point

00:18:01.660 was you know that go is a massively complex space and you know all the traditional methods that

00:18:09.580 have previously been used for games before dqn which essentially involved handcrafting rule-based

00:18:15.900 features which is is really what drove the work behind deep blue ibm's uh model you know a long time

00:18:22.940 ago 97 i think it was you know go has something like 10 to the power of 170 possible configurations

00:18:30.860 of the board so it's a 19 by 19 board with black and white stones and the rules are very simple it's a

00:18:37.420 turn-based game where each player simply moves one place places one stone on the board and when you

00:18:44.780 surround your opponent's stones you remove them from the board and the goal is to sort of surround your

00:18:49.740 opponent and so it is a very simple rule set but it's a massively complicated possible set of different

00:18:58.460 configurations that can emerge and so you you can't sort of search all possible branches of of of that

00:19:05.580 space because it's so enormous yeah i mean 10 to the 170 is like more atoms than there are in the known

00:19:10.780 universe approximately yeah i think that's i think something like 10 to the 80 that gets you all the

00:19:16.540 protons in the universe so yeah there's it gets bigger still when you're talking about though yeah

00:19:22.540 right so you know and so this needed a new suite of methods and you know i think it was an incredible

00:19:29.660 experience seeing alpha go progressively get better and better i mean we we already had an inkling for

00:19:35.260 this when we saw it play the atari games but this was just seismically more complicated and vast and yet it

00:19:42.540 was using the same basic principle actually the same principle that has subsequently been applied

00:19:47.900 in in protein folding too so you know i think that's what's really interesting about this is that is the

00:19:54.300 generality of the ideas that simply scale with more compute you know because a a couple of years later

00:20:01.420 alpha go became alpha zero which essentially achieved superhuman performance without any

00:20:08.060 learn without any learning from prior games so you know part of the trick with alpha go is that it

00:20:14.860 looked at hundreds of thousands of prior games it's almost like the expert knowledge of existing players

00:20:20.380 that has been handed down for centuries of playing the game whereas alpha zero was able to learn entirely

00:20:26.780 through self-play you know almost like i think the intuition is spawning instances of itself in order to play

00:20:33.260 against itself in simulated environments many many hundreds of millions of billions of times way more

00:20:39.980 it turns out to be way more valuable than bootstrapping itself from the first principles of human

00:20:45.580 knowledge which if you think about the size of the state space represents you know a minor subset of all

00:20:51.100 possible configurations of that board and that that was a kind of remarkable insight and and actually

00:20:56.220 it did the same thing for for other games including chess and shogi and so on yeah that's a really

00:21:03.500 fascinating development where it's it's now uncoupled from the repository of human knowledge it plays itself

00:21:10.540 and over the course of i think it was just a day of self-play it was better than than alpha go and any other

00:21:19.500 system right right that's exactly right and and obviously that's partly a function of compute but the

00:21:25.900 basic principle gives an important intuition which is that because these methods are so general they

00:21:31.820 can be paralyzed and scaled up and that means that you know we can sort of take advantage of all of the

00:21:39.340 you know traditional assets of you know computing infrastructure rather than relying on you know old

00:21:44.780 school methods you know perfect memory paralyzable compute you know moore's law you know daisy chaining

00:21:52.140 compute together just like we do with with gpus these days so you know in some ways that's a that's the

00:21:58.780 key intuition because it means the sort of barrier to application of the quality of the algorithm is lower

00:22:06.380 because it's turbocharged by all these other underlying drivers which are also improving the power and

00:22:12.940 performance of these models and um also alpha zero in when it was playing the world champion

00:22:20.700 came up with a move that you know all go experts thought they immediately recognized as a mistake but

00:22:27.740 then when the game played out it turned out to be this brilliant novel move that no human

00:22:33.580 would have made and it just a big piece of discovered go knowledge yeah i mean i i remember

00:22:41.340 sitting in the commentary room live watching that unfold and uh listening to the commentator who was

00:22:48.460 himself a nine down uh you know expert say that it was a mistake he was like oh no it's we lost and uh

00:22:55.660 it took 15 minutes for you know him to correct that and and sort of come back and reflect on it it was a

00:23:01.740 really remarkable moment and actually you know for me it was a great inspiration you know because

00:23:08.540 this is why we we started the company i mean the quest was to try to invent new knowledge i mean our

00:23:16.060 our goal here is to try to design algorithms that can teach us something that we don't know um

00:23:22.220 not just reproduce existing knowledge and synthesize information in new ways but genuinely discover

00:23:27.980 new strategies or new molecules or you know new compounds new ideas and contribute to the you know

00:23:35.980 the kind of well of human knowledge and capability and you know this was a kind of first well actually

00:23:41.900 it was the second indication because the first instinct i got for that was watching the atari games

00:23:46.140 player learn new strategies from scratch and this this was kind of the second i think and what about

00:23:52.060 alpha fold because this is a very different application of the same technology what what

00:23:57.820 what did you guys do there and what was the the project well protein folding is a long-standing

00:24:04.860 challenge and we actually started working on this as a hackathon which started in my group back in 2016

00:24:13.100 and it was really just an experiment to see if you know some of the alpha go models could could

00:24:18.540 actually make progress here and the basic idea is that if you can sort of generate you know an an

00:24:25.900 example of the way a protein folds this folding structure represents might tell you something about

00:24:32.940 you know the the the value of that molecule in practice what it can do what you know what its

00:24:38.620 strengths and weaknesses are and so on and so you know the the nice thing about it is because it

00:24:43.820 operated in a simulated environment it was it was quite similar to some of the games that we had been

00:24:48.540 playing you know teaching our models to to play and you know previously the experiments had done

00:24:55.420 something like 190 000 proteins um which is about 0.1 percent of the of all the proteins in existence but

00:25:03.260 but in the in alpha fold 2 the team actually open sourced something like 200 million protein structures

00:25:09.660 all in one go which is sort of all all known proteins this is a massive breakthrough that took you know

00:25:16.140 four or five years of of work in development and and i think just gives a an indication of the kinds

00:25:22.380 of things that become possible with these sorts of methods yeah i forget someone gave a what purported

00:25:29.100 to be a kind of a straightforward comparison between what alpha fold did there and the academic years of

00:25:37.500 phd theses and it was something like you know 200 million phd theses got accomplished in a few years

00:25:44.620 there uh in terms of solving those protein folding problems yeah i mean that that that's those kinds

00:25:50.700 of insights those kinds of sort of compressions are similar to you know across the board with many

00:25:56.380 technologies like i another one that's sort of similar to that is that the amount of of sort of

00:26:01.740 labor that once produced 50 minutes of of light in the 18th century you know now produces 50 years worth of

00:26:10.060 light and and that just gives a a a sense for how technology has this massive compressive effect

00:26:18.300 that is hugely leveraging in terms of what we can do yeah there's there's another crazy analogy in your

00:26:23.740 book talking about the size of these uh the parameters of these new large language models which

00:26:31.260 we'll get to but the comparison was something like executing all of these floating point operations

00:26:36.540 it's it's if you know if every operation were a drop of water you know the largest large language

00:26:43.340 models execute as many calculations as would fit in to the entire pacific ocean so it's just the scale is

00:26:51.020 is astounding right so your book it was a bit of a surprise for me because you are more worried than i

00:27:00.940 i realized about how all of this can go wrong and i i got the sense in you and i haven't spoken very much

00:27:07.740 but in talking to you and demis and shane i got the sense that and these conversations are you know for now

00:27:15.500 several years old that you were more sanguine about our solving all of the relevant problems you know alignment

00:27:24.540 being the chief among them but other concerns of bad incentives and arms race conditions and

00:27:31.500 etc that you were you all were putting a fairly brave face on on a problem that was making many of us

00:27:40.540 increasingly shrill and you know not to say hysterical um and so there were you know i guess the most

00:27:48.060 hysterical voice of the moment is someone like eliezer yudkowski and there was obviously nick bostrom and

00:27:54.060 others who who were you know issuing fairly grave warnings about how it was more likely than not

00:28:01.260 that we were going to screw this up and build something that we really can't control ultimately

00:28:06.460 and that would could well destroy us and on the way to the worst possible outcome there are many bad

00:28:13.740 very likely outcomes like you know a misinformation apocalypse and other risks but in your book you're

00:28:22.460 you don't give the risks short shrift i mean you're really you do seem to suggest that and certainly

00:28:30.540 when you add in the the attendant risks of synthetic biology here which we'll talk about you are quite

00:28:37.660 worried and yet there's no as you agree with a point i made uh early on here which is that you know as

00:28:46.060 worried as as we are there really is no brake to pull i mean the incentives are such that we're

00:28:51.820 going to build this and so we have to sort of figure out how to repair the the rocket as it's

00:28:57.980 taking off and align it properly as it's taking off because there's just no there's no getting off this

00:29:03.020 ride at the moment despite the fact that people are calling for a moratorium or some people are so i'm

00:29:09.100 just i guess what before we jump into the book when did you get worried were you were you were

00:29:15.020 you always worried or um are you among the newly worried people like jeff hinton who i mean like

00:29:21.020 so just like jeffrey hinton who you who you mentioned is really the godfather of this technology and he just

00:29:28.060 recently resigned from google so that he could express his worries in public and he seems to have just

00:29:34.460 become worried in the presence of these you know large language models and it's quite inscrutable to

00:29:41.500 me that he suddenly had this change of heart because you know in my view the basis for this concern was

00:29:49.740 always self-evident so give me the the memoir of your concerns here yeah so this is not a new consideration

00:29:58.060 for me i i've been worried about this from you know the the very first days when we founded the

00:30:03.340 company and you know in fact our our strap line on our business plan that we took to silicon valley

00:30:10.140 in 2010 was building artificial general intelligence safely and ethically for the benefit of everyone

00:30:16.780 and that was something that was critical to me all the way through when we when we sold the company we

00:30:22.380 made it a condition of the acquisition that we have an ethics and safety board with some independent

00:30:27.340 members overseeing technology in the public interest that we our technologies wouldn't be used for

00:30:32.300 military purposes like lethal autonomous weapons or surveillance by the state you know and and since

00:30:38.940 then at google i you know went through lots and lots of different efforts to experiment with different

00:30:44.540 kinds of oversight boards and charters and external scrutiny and independent audits and all kinds of

00:30:50.380 things and so i'd say i definitely been top of mind for me all the way through i think where i diverge from

00:30:57.340 the sort of bostrom camp a bit is that i think that the language around super intelligence has actually

00:31:04.700 been a bit of a distraction and i think it was quite obviously a distraction from from fairly early on i think

00:31:12.140 that the focus on this you know sort of intelligence explosion this ai that recursively self improves and

00:31:20.220 suddenly takes over everybody and turns the world to paper clips i think has consumed way more time than

00:31:27.100 the idea justifies and actually i think there's a bunch of more near-term very practical things that we

00:31:34.780 should be concerned about they don't they shouldn't create shrill alarmism or panic but they are real

00:31:41.660 consequences that if we don't take them seriously then they have the potential to cause you know serious

00:31:47.420 harm and if if if we continue down this path of complete openness without any you know sort of

00:31:54.380 checks and balances on how this technology arrives in the world then essentially it has the potential

00:32:00.940 to cause a great deal of chaos and i'm not talking about ai's running out of control and you know and

00:32:06.780 robots and so on i'm i'm really talking about you know massively amplifying the spread of of

00:32:12.300 misinformation and more generally reducing the power reducing the barrier to entry to be able to

00:32:18.620 exercise power that that is fundamentally what this technology is i mean in my book i have a framing

00:32:25.180 which i think is more helpful around a modern turing test one that evaluates capabilities like what

00:32:31.820 can an ai do and i think that we should be much more focused on what it can do rather than what it can say

00:32:39.260 right what it can say is important and has huge influence but increasingly it's going to have

00:32:44.140 capabilities and so an artificial capable intelligence an aci is something that has the potential not just

00:32:53.180 to influence and persuade but also to learn to use apis and initiate actions queries calls in third-party

00:33:02.380 environments it'll be able to use browsers and parse the pixels on the browser to be able to click

00:33:09.180 buttons and take actions in those environments it'll be able to call you know phone up and speak to

00:33:15.660 communicate with other ai's and other humans so you know these technologies are getting smaller and

00:33:22.940 smaller and more and more capable are getting cheaper to build and so if you look out over a 10 to

00:33:28.940 20 year period i think the story is one of a proliferation of power in the conventional sense

00:33:35.660 not so much an intelligence explosion which by the way just for the record i think is an important

00:33:40.940 thing for us to think about and i care very deeply about existential risk and agi safety but i think that

00:33:48.140 the more practical risks are not getting enough consideration and that's actually a big part of the

00:33:53.500 book in no way does that make me a pessimist i mean i'm absolutely an optimist i'm hopeful and positive

00:34:00.700 about technology i want to build things to make you know people's lives better and to help us create

00:34:05.580 more value in the world and and reduce suffering and i think that's the true upside of these technologies

00:34:10.860 and we will be able to deliver them on that upside but no technology comes without risk and and we have

00:34:16.700 to consciously and proactively you know attend to the downsides you know otherwise you know we haven't

00:34:24.860 really achieved our full objective and that's the purpose of speaking up about it well before we get

00:34:30.700 into details about the downsides let's talk about how this might go well i guess before we talk about

00:34:39.260 the upside let's just define the terms in the title of your book the title is the coming wave what is

00:34:45.740 the coming wave so when you look back over the millennia there have been waves of general purpose

00:34:53.740 technologies from fire to the invention of the wheel to electricity and each of these waves to the

00:35:01.500 extent that they have been lasting and valuable are general purpose technologies which enable other

00:35:07.020 technologies and that's what makes them a wave they're enablers of other activity their general purpose

00:35:12.460 in nature and as they get more useful naturally people experiment with them they iterate they invent

00:35:20.540 they adapt them and they get cheaper and easier to use and that's how they proliferate so in the

00:35:27.740 history of technologies all technologies that have been useful that are real general purpose technologies

00:35:33.420 have spread far and wide and got cheaper and almost universally that is an incredibly good thing it has

00:35:41.180 transformed our world and i think that that's an important but very simple concept to grasp because

00:35:50.940 if that is a law of technology if it is a fundamental property of the evolution of technology which i'm

00:35:57.980 arguing it is then that has real consequences for the next wave because the next wave is a wave of

00:36:05.660 intelligence and of life itself right so intelligence is the ability to take actions it is the ability to

00:36:16.460 synthesize information make predictions and affect the world around you so it's almost the definition of of

00:36:24.300 power and everything that is in our visual sphere everything in our world if you look around you at this very

00:36:30.700 minute today has been affected in a very material way by intelligence it is the thing that has produced

00:36:37.980 all of the value and all of the products and all of the you know affected the landscape that you can see

00:36:43.260 around you in a huge way and so the prospect of being able to distill what makes us unique as a species

00:36:52.220 into an algorithmic construct that can benefit from being scaled up and paralyzed that can benefit from

00:36:59.740 perfect memory and compute and consuming vast amounts of of data trillions of words of data is is enormous i

00:37:06.940 mean that that in itself is is almost like you know gold it's it's like being able it's like alchemy it's it's

00:37:14.380 like being able to capture the essence of what has made us capable and add more knowledge and you know essentially

00:37:22.300 the science and technology into the the human ecosystem so imagine that everybody will now in

00:37:29.020 the future in 10 years 15 years have access to the very best you know doctor in the world the very best

00:37:37.820 educator you know the very best personal assistant and chief of staff and any one of these roles

00:37:43.900 i think is going to be very very widely available to billions of people you know people often say to

00:37:49.660 me well you know isn't aren't the rich going to benefit first or is it going to be unfair in

00:37:54.540 terms of access you know yes for a period of time that's true but we're actually living in one of the

00:38:00.380 most meritocratic moments in the history of our species every single one of us no matter how wealthy

00:38:06.860 you are every one of us in the western world really the top two billion people on the planet

00:38:12.860 have access to the same smartphone right no matter how much you earn you cannot buy a smartphone or a

00:38:19.740 laptop that is better than the very richest that's an unbelievably meritocratic moment that is worth really

00:38:27.020 meditating on and that is largely a function of these exponentials you know the cost of chips has

00:38:34.140 exponentially declined over the last 70 years and that's driven mass proliferation and if intelligence

00:38:41.980 and life are subject to those same exponentials which i think they are over the next two to three decades

00:38:49.260 then the primary trend that we have to cope with in terms of our culture and our politics and commerce

00:38:56.220 is this idea that intelligence the ability to get stuff done

00:38:59.980 is about to proliferate and that's going to produce a a cambrian explosion of productivity

00:39:07.660 everybody is going to get access to a tool that enables them to pursue their agenda to make us all

00:39:13.180 smarter and more productive and more capable so i i think it might be one of the most productive

00:39:18.860 periods in in the history of of humanity and and i think of course the challenge there is that it may

00:39:25.500 also be one of the most unstable over the next 20 years yeah so that um cornucopia image immediately

00:39:35.660 begets the downside concern of massive labor disruption which um many people doubt in principle they just

00:39:44.780 think that we've we've learned over the course of the last 200 years of technological advancement and

00:39:50.860 economic thinking that there is no such thing as a true canceling of a need for human labor

00:39:59.580 and so people draw the obvious analogies from agriculture and other you know previous periods of

00:40:05.340 labor disruption and conclude that this time is no different and we will while there might be a few

00:40:14.140 hiccups what's going to happen here is that all of these productivity gains and job canceling innovations born of ai

00:40:23.660 will just open new lanes for a human creativity and there'll be better jobs and you know we're just as

00:40:32.860 we were happy to get rid of jobs in agriculture and coal mines and open them up in in the service sector

00:40:40.540 we're going to do the same with ai i remain quite skeptical of uh that this time is the same given

00:40:49.420 the nature of the technology and this is this is the first as you just said this is the first moment

00:40:54.220 where we are envisioning a technology which is a true replacement for

00:41:02.300 human intelligence at every move if we're talking about general intelligence and we're talking about

00:41:08.220 the competence that you just described the ability to do things in addition to saying things

00:41:15.580 where we are talking about the cancellation of uh human work at least in principle and you know

00:41:24.540 strangely i mean this is not a not a terrible surprise now but it would have been a surprise

00:41:29.580 probably 20 years ago this is coming for the higher cognitive higher status white-collar jobs before

00:41:36.940 it's coming for blue-collar jobs how do you view the prospect of labor disruption here and how confident

00:41:46.700 are you that everyone can be um retrained with their um nearly omniscient ai assistants and chiefs of

00:41:55.740 staffs and uh find something worth doing that other people will pay them to do yeah i mean i i'm with

00:42:03.020 you i've i've long been skeptical of people who've said that you know this will be just like the

00:42:09.100 agricultural revolution or you know this will be like the horse and cart and and and cars you know

00:42:15.260 people will have more wealth the productivity will will drive wealth creation and then that wealth

00:42:21.820 creation will drive demand for new products and we couldn't possibly imagine you know what people are

00:42:26.620 going to want to consume and and what people are going to create with this new wealth and new time and

00:42:31.580 and that that's typically how the argument goes and i i've never found that compelling i mean i i think

00:42:38.860 that if you if you look at it it's been quite predictable the last decade i mean these models

00:42:44.860 are deliberately trying to replace human cognitive abilities in fact they have been slowly climbing the

00:42:52.860 ladder of of of human cognitive abilities for many years i mean we started with image recognition

00:42:58.940 um and audio recognition and then moved on to you know audio generation image generation and then text

00:43:07.660 you know understanding text recognition and then now text generation and you know it was kind of

00:43:14.620 interesting because if you think even just two or three years ago people would have said well ai's will

00:43:21.260 will will never be creative that's not achievable you know that that creativity will always be the

00:43:27.420 preserve of of humans and and and judgment is somehow unique and special to what it means to

00:43:32.540 be human or like you know ais will never have empathy will always be able to do care work and you know

00:43:37.980 emotional care is is something that's special you can never replace that connection i mean both of

00:43:42.700 those are now self evidently not not true and i think have been quite predictable so i think that the

00:43:48.620 the way to the honest way to look at this is that these are only temporarily augmenting of human

00:43:53.820 intelligence if you think about the trajectory over 30 years i mean let's not quibble over whether it's

00:43:58.700 five years 10 years or 15 years just think about it long term i think we can all agree long term if

00:44:03.100 these exponential trajectories continue then you know they're they're clearly only temporarily

00:44:10.140 going to turbocharge an existing human and so we have to really think okay long term what does it mean

00:44:16.860 to have systems that are this powerful this cheap this widely proliferated and that's where i think

00:44:23.820 the the broad concept i have in the book of containment comes in because you can start to

00:44:27.580 get an intuition for you know the the massive consequences of the spread of this kind of power

00:44:33.740 and then start to think about what are the what are the sorts of things we would want to do about

00:44:37.340 it because on the face of it like you said earlier the incentives are absolutely overwhelming i mean

00:44:42.780 technology has always been a machine of of of statecraft it's been used by militaries and used

00:44:48.940 by nation states to serve citizens and drive us forward and now it is the fundamental driving force

00:44:56.540 of nation states you know being commercially competitive having the best companies having the

00:45:01.180 best labor market that drives our competitive edge you know so from a state perspective a nation state

00:45:07.420 perspective you know from an individual scientific perspective the huge drive to explore and invent

00:45:13.420 and discover and of course from a commercial perspective the you know the the profit incentive

00:45:19.020 is is phenomenal and all of these are good things provided they can be well managed and provided we can

00:45:25.260 mitigate the downsides and i think we have to be focused on those downsides and not be afraid to talk about

00:45:31.020 them i mean you know so i definitely experience when i bring up these topics over over the years this

00:45:37.980 kind of what i describe in the book as a pessimism aversion you know there's there's people who are

00:45:42.700 just just sort of constitutionally unable to have a dark conversation about how things may go wrong

00:45:49.740 and i'll get accused of like not being an optimist or something as though that's like a you know a sin or

00:45:55.260 something or or or that being a pessimist or an optimist is somehow you know a good way of framing

00:46:01.260 things to me both are biased i'm just observing you know the the kind of facts as i see them and i

00:46:07.020 think that's an important sort of misconception and unhelpful framing of pessimism and optimism

00:46:12.380 because we have to start with our best assessment of the facts and try to reject those facts if they're

00:46:17.980 you know inaccurate in some way and then then try to collectively predict what the consequences are

00:46:22.700 going to be like and i think it's sort of another trend over the last sort of decade or so people

00:46:27.260 you know post-financial crisis i feel like people public intellectuals and elites in general and

00:46:32.860 everyone in general has sort of just like got a bit allergic to predictions right we've got a bit

00:46:37.740 scared of being wrong and i think that that's another thing that we've got to shed so we've got to focus

00:46:42.860 on trying to make some of these predictions they may be wrong you know i may have got this completely wrong

00:46:47.340 but it's important to lay out a case for what might happen and start taking steps towards you know

00:46:54.780 mitigation and adaptation well you invoke the concept of containment which does a lot of work

00:47:01.260 in the book and you have this phrase the containment problem that you use throughout what is the

00:47:08.460 containment problem in its most basic form the idea of containment is that we should be able to

00:47:15.260 demonstrate to ourselves that technologies that we invent should always be accountable to humans and

00:47:23.420 within our control so it's the ability to close down or constrain or limit a new technology at any

00:47:30.540 stage of its development or deployment and you know that's that's a grand claim but actually put in

00:47:37.180 the most simple terms it basically says we shouldn't allow technologies to run out of our control

00:47:43.580 right if we can't say what destiny we want for how a technology impacts our species then we're at the

00:47:50.700 mercy of it right and i think i think the the net the the idea is if if we if we don't have mechanisms to

00:47:57.420 shape that and restrict its capabilities then it potentially leads us into some some quite catastrophic

00:48:04.780 outcomes over a 30 or 30 year period do you think we've lost the moment already i mean it seems like

00:48:13.740 the the digital genie is is more or less out of the bottle i mean this is something that i mean if

00:48:18.940 anything surprised me and i know certainly surprised the people who are were more focused on on ai safety

00:48:26.140 and again people like utkowski in recent developments around these llms was that we missed a moment that

00:48:33.580 many of us more or less expected or more or less sure was coming which was there'd be a breakthrough

00:48:39.900 at some company like deep mind where we would you know the people building the technology would

00:48:46.300 recognize that they had finally gotten into the end zone or close enough to it so that they're

00:48:52.380 now in the presence of something that's fundamentally different than anything that's come before and

00:48:57.980 there'd be this question okay is this safe to work with is it is this safe to release into the wild is

00:49:05.820 this safe to create an api for is to say and you know so this the the idea was that you'd have this

00:49:12.620 you know digital oracle you know in in a box that would be um you know would already have been air

00:49:18.860 gapped from the internet and incapable of doing anything until we let it out and then the question would be

00:49:24.620 have we have we done enough safety testing to let it out but now it's pretty clear that everything is

00:49:30.620 already more or less out and we're building our most powerful models already in the wild right and

00:49:38.300 they're already hooked up to things and they're they already have millions of people playing with

00:49:42.540 them and they're they're open source versions of the next best model and so is containment even a

00:49:49.260 dream at this point so it's definitely not too late we're a long long way away this is really

00:49:55.180 just the beginning uh you know we we have plenty of time to address this and the more that these models

00:50:03.100 and these ideas happen in the open the more they can be scrutinized and they can be pressure tested

00:50:09.660 and held accountable so i think it's great that they're happening in open source at the moment so

00:50:14.620 you like sam altman's this is what sam has always said that the the philosophy behind open ai is do

00:50:21.820 this stuff out in the open let people play with it and we will learn a lot as we get closer and

00:50:29.100 closer to building something that we have to worry about i i think that we have to be humble about the

00:50:35.580 practical reality about how these things emerge right so the initial framing that it was going to be

00:50:42.620 possible to invent this oracle ai that stays in a box and we'll just probe it and poke it and test

00:50:48.300 it until we can prove that it's you know going to be safe and that we'll stay in the bunker and keep

00:50:54.140 it hidden from everybody i mean this is a complete nonsense and it's attached to the super intelligence

00:50:58.940 framing it was just a completely wrong metaphor that totally ignores the history of all technologies

00:51:04.780 and actually this is one of the core motivations for me in the book is that i had time during the

00:51:08.540 pandemic to really like you know sleep and reflect and really deeply think okay what is actually

00:51:14.140 happening here on a multi-century scale and what are the patterns of history uh around how inventions

00:51:21.260 end up proliferating and it's really stating the obvious it's almost like ridiculously simplistic but

00:51:27.180 it needed to be said that actually as soon as something as an idea is invented millions of other people

00:51:34.380 people have approximately the same idea within just weeks months years especially in our modern digitized

00:51:41.900 world and so we should expect and as we do see the open source movement to be right hot on the heels

00:51:49.420 of the absolute frontier and so i mean just one small example of that to give an intuition gpt3

00:51:57.420 it was launched in the summer of 2020 so three years ago 175 billion parameters and is now regularly being

00:52:05.500 trained at two billion parameters and so that is a massive reduction in serving cost you know that now

00:52:14.060 means that people can have open source versions of gpt3 that have broadly the same capabilities right but

00:52:21.820 are actually extremely cheap to serve and indeed to train so if that trajectory continues then we should

00:52:29.740 expect that what is cutting edge today frontier models like arza inflection and and like gpt4 gpt3.5

00:52:39.100 even will be open source in the next two to three years and so what does it mean that those capabilities

00:52:45.260 are available to everybody right and i think that is a great thing for where we are today but if the

00:52:50.300 trajectory of exponentially increasing compute and size of models continues for another three four

00:52:57.660 five generations which we all expect it to then that's a different question we have to step back

00:53:03.020 and honestly ask ourselves what does it mean that this kind of power is going to proliferate proliferate

00:53:07.500 in open source number one and number two how do we hold accountable those who are developing these mega

00:53:13.180 models even if they are centralized and closed myself included open ai deep mind etc and if you just look at

00:53:19.660 the amount of compute it's predictable and breathtaking and i think people forget how predictable this is

00:53:26.060 so going back to atari dqn we developed that model in 2013 and it used two petaflops of computation

00:53:36.300 right so a petaflop is a billion million operations right so imagine a billion people each holding one million

00:53:45.820 calculators each and doing a complex calculation all at the same time pressing equals right so that's

00:53:53.100 that would be one petaflop and atari used two petaflops over several weeks of computation a decade later the

00:54:01.180 cutting-edge models that we develop at inflection for pi our ai use five billion times the compute that was used to

00:54:09.820 play atari dqn so 10 billion billion million um it's just like now you're sounding like la g exactly

00:54:22.060 that's that's that's basically 10 orders of magnitude more compute in a decade so one order of magnitude

00:54:28.620 every year so 10 x every year for 10 years which is way more than moore's law everyone's familiar with

00:54:36.140 moore's law 70 years of doubling doubling every 18 months or whatever i mean that's that is minuscule

00:54:42.060 by comparison now of course there's a very good hand if you'd like to continue listening to this

00:54:47.180 conversation you'll need to subscribe at sam harris.org once you do you'll get access to all

00:54:52.540 full-length episodes of the making sense podcast along with other subscriber only content including

00:54:57.740 bonus episodes and amas and the conversations i've been having on the waking up app the making sense

00:55:03.500 podcast is ad free and relies entirely on listener support and you can subscribe now at sam harris.org

Making Sense - Sam Harris - August 28, 2023

#332 — Can We Contain Artificial Intelligence？

Episode Stats

Length

Words per Minute

Word Count

Sentence Count

Hate Speech Sentences

Summary

Transcript