Making Sense - Sam Harris - #324 — Debating the Future of AI

00:00:00.000 welcome to the making sense podcast this is sam harris just a note to say that if you're hearing

00:00:12.480 this you are not currently on our subscriber feed and will only be hearing the first part

00:00:16.880 of this conversation in order to access full episodes of the making sense podcast you'll need

00:00:21.920 to subscribe at sam harris.org there you'll find our private rss feed to add to your favorite

00:00:27.000 podcatcher along with other subscriber only content we don't run ads on the podcast and

00:00:32.480 therefore it's made possible entirely through the support of our subscribers so if you enjoy

00:00:36.500 what we're doing here please consider becoming one

00:00:38.800 okay well there's been a lot going on out there everything from elon musk and mark zuckerberg

00:00:54.680 challenging one another to an mma fight which is ridiculous and depressing to robert kennedy jr

00:01:04.040 appearing on every podcast on earth apart from this one i have so far declined the privilege

00:01:10.940 it really is a mess out there i'll probably discuss the rfk phenomenon in a future episode

00:01:18.140 because it reveals a lot about what's wrong with alternative media at the moment but i will leave

00:01:25.440 more of a post-mortem on that for another time today i'm speaking with mark andreessen mark is a

00:01:32.380 co-founder and general partner at the venture capital firm andreessen horowitz he's a true internet

00:01:37.600 pioneer he created the mosaic internet browser and then co-founded netscape he's co-founded other

00:01:44.440 companies and invested in too many to count mark holds a degree in computer science from the

00:01:49.580 university of illinois and he serves on the board of many andreessen horowitz portfolio companies

00:01:54.320 he's also on the board of meta anyways you'll hear mark and i get into a fairly spirited debate

00:02:01.140 about the future of ai we discuss the importance of intelligence generally and the possible good

00:02:07.380 outcomes of building ai but then we get into our differences around the risks or a lack thereof of

00:02:13.600 building agi artificial general intelligence we talk about the significance of evolution in our

00:02:19.920 thinking about this the alignment problem the current state of large language models how developments

00:02:26.800 in ai might affect how we wage war what to do about dangerous information regulating ai economic

00:02:35.460 inequality and other topics anyway it's always great to speak with mark we had a lot of fun here

00:02:42.060 i hope you find it useful and now i bring you mark andreessen

00:02:46.860 i am here with mark andreessen mark thanks for joining me again

00:02:56.100 it's great to be here sam thanks i got you on the end of a swallow of some delectable beverage

00:03:01.740 yes you did so um this should be interesting i'm i'm uh eager to speak with you specifically

00:03:08.940 about this recent essay you wrote on ai and so you obviously many people have read this and you are a

00:03:16.780 voice that um many people value on this topic among others perhaps you've been on the podcast before

00:03:23.820 and people know who you are but maybe you can briefly summarize uh how you come to this question i

00:03:31.740 mean what how would you summarize your um the relevant parts of your career with respect to the question of ai and its

00:03:38.540 possible ramifications yeah so i've been a computer programmer technologist computer scientist since the

00:03:45.020 1980s when i actually entered college in 1989 at university of illinois the ai field had been through a boom in the

00:03:51.580 80s um which had crashed hard and so by the time i got to college it was the ai wave was was was dead

00:03:58.300 and buried at that point for a while um it was like the backwater of the department that nobody really

00:04:02.620 wanted to talk about and then but you know i learned i learned a lot of it i learned a lot of it in a

00:04:06.620 school and then uh i went on to you know help help create what is now kind of known as the modern

00:04:11.660 internet in the 90s and then over time transitioned to become a went from being a technologist to being

00:04:17.580 an entrepreneur and then today i'm an investor venture capitalist and so 30 years later 30 35

00:04:22.300 years later i'm involved in a very broad cross-section of tech companies that uh have many

00:04:28.540 many of them have have many kind of ai aspects to them and so you know and everything from facebook now

00:04:34.380 meta you know which has been investing deeply in ai for over a decade yeah through to many of the best

00:04:38.700 new ai startups you know we're our day our day job is to find the best new startups in a new category

00:04:43.980 like this and try to back the entrepreneurs and so that's a that's how i spend most of my time right

00:04:47.900 now okay so the the essay is titled why ai will save the world and i think even in the title alone

00:04:54.940 people will detect that you are striking a different note than i tend to strike on this topic i i think

00:05:00.860 we'll yeah i disagree with a few things in the essay that are i think at the core of my interest here but

00:05:08.460 i think there are many things you know we agree about you know up front we agree i think with more

00:05:14.540 or less anyone who thinks about it that intelligence is good and we want more of it and um i mean if it's

00:05:20.380 not necessarily the source of everything that's good in human life it is what will safeguard everything

00:05:26.060 that's good in human life right so even if you think that love is more important than intelligence and

00:05:30.860 you think that playing on the beach with your kids is way better than doing science or anything

00:05:35.340 else that is narrowly linked to intelligence well you have to admit that you value all of the things

00:05:42.620 that intelligence will bring that will safeguard the things you value i mean so a cure for cancer

00:05:47.900 and a cure for alzheimer's and a cure for a dozen other things will give you much more time with the

00:05:52.620 people you love right so whether you think about the primacy of intelligence or not very much it is the

00:05:58.540 thing that that has differentiated us from our primate cousins and it's the thing that allows us to do

00:06:03.660 everything that is maintaining the status of civilization and if the future is going to be

00:06:09.580 better than the past it's going to be better because of what we've done with our intelligence

00:06:13.740 in some basic sense and because i think you're we're going to agree that because intelligence is so good

00:06:19.420 and because each increment of it is good and profitable this ai arms race and gold rush is not

00:06:26.460 going to stop right we're not going to pull the brakes here and say let's take a pause of five years

00:06:31.500 and not build any ai right i mean i think that's i don't remember if you you address that specifically

00:06:36.860 in your essay but even if some people are calling for that i don't think that's in the cards i don't

00:06:41.980 think you think that's in the cards well there are you know it's hard it's hard to believe that you

00:06:46.300 just like put in the box right and stop working on it it's hard to believe that the progress stops

00:06:50.460 you know like having said that there are some powerful and important people who are in washington

00:06:53.500 right now advocating that and there are some politicians who are taking them seriously so they're

00:06:57.340 there you know they're at the moment there is some danger around that and then look there's two

00:07:01.260 other big dangers two other scenarios that i think would both be very very devastating for the future

00:07:05.900 one is the scenario where the fears around ai are used to basically entrench a cartel so and and this

00:07:12.060 is what this is what's happening right now this is what's being lobbied for right now is there are

00:07:15.180 a set of big companies that are arguing in washington yes ai you know has positive cases uses yes ai is

00:07:21.740 also dangerous because it's dangerous therefore we need a regulatory structure that basically entrenches a set

00:07:26.700 of currently powerful tech companies you know to be able to have basically exclusive rights to

00:07:32.220 to to do this technology i think that would be devastating for reasons we could discuss and then

00:07:36.220 look there's a third outcome which is we lose china wins right they're certainly working on ai

00:07:40.700 and they have a you know what what i would consider to be a very dark and dystopian vision of the future

00:07:45.340 which i also do not want to win yeah i mean i guess that that is in part the cash value of the

00:07:50.940 point i just made that even if we decided to stop not everyone's going to stop right i mean

00:07:55.100 human beings are going to continue to grab as much intelligence as we can grab even if in some

00:08:01.260 local spot uh we decide to pull the brakes although it really is at this point it's hard to imagine

00:08:08.380 even whatever the regulation is it really stalling progress i mean given just again given the the

00:08:15.980 intrinsic value of intelligence and given the excitement around it and given the obvious dollar

00:08:21.500 signs that everyone is seeing i mean the incentives are such that i just don't see it but well we'll

00:08:27.180 come to the the regulation piece eventually because i think it's i you know i i given the difference in

00:08:33.020 our views here i i it's not going to be a surprise that i want some form of regulation and i'm not quite

00:08:38.060 sure what that could look like and i think you have a bet you would have a better sense of uh what it

00:08:43.500 looks like and perhaps why you're um that's why you're worried about it but um before we talk about

00:08:50.220 the fears here let's talk about the good outcome because you you sketch a fair i mean i know you

00:08:55.820 don't consider yourself a utopian but you sketch a fairly utopian picture of promise in your essay

00:09:02.620 if we got this right what how good do you think it could be yeah so let's just start by saying i i kind

00:09:09.180 of deliberately loaded my let the title of the essay with a little bit of a religious element and i did

00:09:13.180 that kind of very deliberately because i i view that i'm up against a religion the sort of ai risk

00:09:17.260 fear religion so but but i i am not myself religious you know lowercase are religious in the sense of

00:09:22.780 you know i'm not i'm not a utopian i'm very much uh yeah i'm an adherent to what thomas solo called

00:09:26.620 the constrained vision not the unconstrained vision so i'm i'm i live i live in a world of

00:09:29.900 practicalities and trade-offs and so yeah i'm i am actually not utopian look having said that

00:09:35.580 building on what you've already said like intelligence if there is a lever for human progress across

00:09:40.380 many thousands of domains simultaneously it is intelligence and we just we know that because

00:09:44.300 we have thousands of years of experience seeing that play out the thing i would add to i thought

00:09:48.460 you you made that case very well the thing i would add to the case you made about the positive virtues

00:09:52.860 of intelligence in human life is that the way you described it at least the way i heard it was more

00:09:58.140 focused on like the social societal wide benefits of intelligence for example cures for diseases and so

00:10:02.460 forth that that is true and i agree with all that there are also individual level benefits of

00:10:07.020 intelligence right at the level of an individual even if you're not the scientist who invents a cure

00:10:10.860 for cancer at an individual level if you are smarter you have better life welfare outcomes on

00:10:15.980 almost every metric that we know how to measure everything from how long you'll live how healthy you'll

00:10:20.300 be how much education you'll achieve career success the success of your children by the way

00:10:25.420 your ability to solve problems your ability to deal with conflict smarter people are less violent

00:10:30.780 smarter people are less bigoted and so there's this very broad kind of pattern of human behavior

00:10:36.380 where basically more intelligence you know just simply at the individual level leads to better

00:10:40.060 outcomes and so the the sort of most you most utopian i'm willing to get is sort of this potential

00:10:46.140 which i think is very real right now it's already started where you you basically just say look

00:10:50.460 human beings from here on out are going to be are going to have an augmentation and the augmentation is

00:10:54.940 going to be in the long tradition of augmentations like everything from eyeglasses to shoes to work

00:10:58.940 processors to search engines but now the augmentation is intelligence and that augmented

00:11:04.380 intelligence capability is going to let them capture the gains of individual level intelligence you

00:11:09.660 know potentially considerably above you know where that where they punch in as as individuals and

00:11:14.860 what's interesting about that is that can scale all the way up right like you know somebody who is

00:11:20.220 you know somebody who struggles with you know daily challenges all of a sudden is going to have a

00:11:24.140 a partner and an assistant i'm going to coach and a therapist and a mentor to be able to help

00:11:28.300 improve a variety of things in their lives and then you know look if you had given this to einstein

00:11:33.260 you know he would have been able to discover a lot more new fundamental laws of physics

00:11:36.780 right in the in the out you know in the in the full in the full uh in the full vision and so you

00:11:40.700 know that this is one of those things where it could help everybody and then it could help everybody

00:11:44.860 in many many different ways hmm yeah we'll see in your essay you go into some detail of

00:11:50.780 bullet points around this concept of everyone having a essentially a digital oracle in their pocket where

00:11:56.220 there that you have this personal assistance who you're you can be continuously in dialogue with

00:12:02.860 and it's just it'd be like having the smartest person who's ever lived just giving you a bespoke

00:12:10.780 concierge service to you know all manner of uh tasks and and you know across any information

00:12:16.540 landscape and i i just i happen to recently want re-watch the film her which i hadn't seen since

00:12:23.580 it came out so it came out 10 years ago and i i don't know if you've seen it lately but it i must

00:12:27.740 say it lands a little bit differently now that we're on the cusp of this thing and while it's not

00:12:35.180 really dystopian there is something a little uncanny and quasi bleak around even the the happy vision

00:12:44.220 here of having everyone siloed in their interaction with an ai i mean it is it's the personal assistant

00:12:51.820 you know in your pocket that becomes so compelling uh and so aware of your goals and aspirations and

00:12:59.740 what you did yesterday and and the email you sent or forgot to send and it you know it's a part from

00:13:05.660 the ending which is kind of clever and surprising and you know he's kind of irrelevant for for our purposes

00:13:11.100 here it's not a it's not an aspirational vision of the sort that you sketch in your essay and i'm

00:13:16.700 wondering even if if you see any possibility here that even the best case scenario has something

00:13:24.380 intrinsically alienating and troublesome about it yeah so look on the movie you know as peter

00:13:30.860 teals pointed out like hollywood no longer makes positive movies about technology

00:13:34.940 so and then look you know he argues it's because they hate you know they hate technology but i would

00:13:39.260 argue maybe a simpler explanation which is they want dramatic tension and conflict

00:13:43.340 right and so they necessarily you know it's going to have things are going to have a dark tinge

00:13:47.260 you know regardless you know they obviously spring-loaded by their choice of character and and

00:13:50.860 and so forth right the scenario i have in mind is actually quite a bit different and and and and

00:13:54.620 let me get kind of maybe philosophical for a second which is you know there's this long

00:13:58.540 running debate this question that you just raised is a question that goes back to the industrial

00:14:01.980 revolution and remember it goes back to the core of actually the you know original marx you

00:14:06.460 know marx's original theory marx's original theory was industrialization technology

00:14:10.700 modern economic development right alienates the human being right from from society right that

00:14:15.020 that was that was his core indictment of technology and and look like there are you can point to many

00:14:20.380 many cases in which i think that has actually happened like i think alienation is a real problem

00:14:24.140 i you know i don't think that critique was entirely wrong his prescriptions were disastrous but

00:14:27.660 i don't think the critique was completely wrong look having said that then it's a question of like okay

00:14:31.020 now that we have the technology that we have and we have you know new technology we can invent like

00:14:35.100 how could we get to the other side of that problem and so i i would i would put the shoe on the other

00:14:39.020 foot and i would say look the the purpose of human existence and the way that we live our lives should

00:14:44.540 be determined by us and it should be determined by us to maximize our potential as human beings

00:14:50.620 and the way to do that is precisely to have the machines do all the things that they can do so that

00:14:55.420 we don't have to right and and and this is why marx ultimately his critique was actually in the long run

00:15:00.300 i think has been judged to be incorrect which is we we are all much better anybody in the developed

00:15:04.620 west you know industrialized west today is much better off by the fact that we have all these

00:15:07.900 machines that are doing everything from making shoes to harvesting corn to doing everything you

00:15:12.300 know so many other you know industrial processes around us like we just have a lot more time and a

00:15:17.020 much more pleasant you know day-to-day life you know than we would have if we were still doing things

00:15:21.740 the way that things used to be done the potential with ai is just like look take take take take the

00:15:26.220 drudge work out like take the remaining drudge work out take all the you know look like i'll give

00:15:30.620 you a simple example office work that you know the inbox staring at you in the face of 200 emails

00:15:36.060 right at friday at three in the afternoon like okay no more of that right like we're not going to

00:15:40.220 do that anymore because i'm going to have an ai assistant the ai assistant is going to answer the

00:15:43.180 emails right and in fact what's going to happen is my ai assistant is going to answer the email that

00:15:47.260 your ai assistant set right it's mutually assured destruction yeah exactly but like the machine should be

00:15:54.060 doing that like the human being should not be sitting there when it's like sunny out and his

00:15:57.500 like you know my when you're my eight-year-old wants to play i'm not i shouldn't be sitting there

00:16:00.460 doing emails i should be out my eight-year-old there should be a machine that does that for me

00:16:03.420 and so i view this very much as basically apply the machines to do the drudge work precisely so

00:16:08.540 that people can live more human lives now this is philosophical people have to decide what

00:16:12.540 kind of lives they want to live and again i'm not a utopian on this and so there's a long

00:16:15.820 discussion we could have about how this actually plays out but that potential is there for sure

00:16:19.660 right right okay so let's jump to the bad outcomes here because this is really why i want to talk

00:16:24.700 to you in your essay you you list five and i'll just read your section titles here and and then we'll

00:16:31.100 take a whack at them the first is uh will ai kill us all number two is will ai ruin our society

00:16:38.540 number three is will ai take all our jobs number four is will ai lead to crippling inequality

00:16:45.020 and five is will ai lead to people doing bad things and i would tend to bin those in in really

00:16:50.860 two buckets the first is will ai kill us and uh that's the existential risk concern and the others

00:16:58.140 are more the ordinary bad outcomes that we tend to think about with other technology you know bad

00:17:04.220 people doing bad things with powerful tools unintended consequences disruptions to the labor market which

00:17:10.460 i'm sure we'll talk about and those are all of those are certainly the near-term risks and uh in

00:17:17.580 some sense even more interesting to people because the the existential risk component is longer term and

00:17:23.020 it's even purely hypothetical and you seem to think it's purely fictional um and this is where i think

00:17:30.300 you and i disagree so let's start with this question of of will ai kill us all and they and the

00:17:36.860 thinking on this tends to come under the banner of the problem of ai alignment right and the concern

00:17:44.380 is that we can build if we build machines more powerful than ourselves more intelligent than

00:17:50.060 ourselves it seems possible that the the space of all possible more powerful super intelligent

00:17:55.980 machines includes many that are not aligned with our interests and not disposed to continually track

00:18:02.700 our interests and many more of that sort than of the sort that perfectly hew to our interests in

00:18:08.220 perpetuity so the concern is we could build something powerful that is essentially an angry

00:18:13.820 little god that we can't figure out how to placate uh once we've built it uh and certainly we don't want

00:18:19.900 to be negotiating with something more powerful and intelligent than ourselves and the picture

00:18:24.140 here is of something like you know a chess engine right we've built chess engines that are more

00:18:29.100 powerful than powerful than we are at chess and once we built them if everything depended on our

00:18:35.500 beating them in a game of chess we wouldn't be able to do it right because they they are simply

00:18:39.900 better than we are and so now we're building something that is that is a general intelligence

00:18:44.860 and it will be better than we are at everything that goes by that name or such as a concern and in

00:18:51.420 your essay i mean i think there's an ad hominem piece that i think we should blow by because you you've

00:18:57.420 already described this as a as a religious concern and you and in the essay describe it as a just a

00:19:03.500 symptom of superstition and that the people are essentially in a in a new doomsday cult and there's

00:19:09.660 some share of true believers here and there's some share of you know ai safety grifters and i think you

00:19:15.740 know i'm sure you're right about some of these people but we should acknowledge up front that there are

00:19:20.700 many super qualified people of of high probity who are prominent in the field of ai research who are

00:19:28.460 part of this chorus voicing their concern now i mean you've got somebody like jeffrey hinton who

00:19:34.300 arguably did as much as anyone to create the breakthroughs that have given us these these llms

00:19:39.580 we have stuart russell who literally wrote the most popular textbook on ai so there are other serious

00:19:45.580 sober people who are very worried for reasons of the sort that i'm going to going to express here

00:19:50.620 so that's i mean that's just i just want to acknowledge that both are true there's the

00:19:55.660 crazy people the the the new millennialists the doomsday preppers the neuro atypical people who

00:20:02.780 are in their polyamorous cults and uh you know ai alignment is their primary fetish but there's a lot

00:20:09.180 of sober people who are also worried about this would you would you acknowledge that much yeah although it's

00:20:14.060 tricky because smart people also have a tendency to fall into cults so it doesn't get you totally off

00:20:20.620 the hook on that one but i i would i would register a more fundamental objection to uh what i would

00:20:25.260 describe as and this is not i'm not knocking you on this but it's something that something that people

00:20:29.500 do is sort of argument by authority i don't think applies either and yeah well i'm not making that yet

00:20:35.180 no i know but like this idea this idea which is very good again i'm not characterizing your idea i'll

00:20:39.020 just say it's a general idea this general idea that there are these experts and these experts are

00:20:43.180 experts because they're the people who created the technology or originated the ideas or implemented

00:20:46.860 the systems therefore have sort of special knowledge and insight in terms of their you know downstream

00:20:51.500 impact on society and rules and regulations and so forth and so on that assumption does not hold up

00:20:57.020 well historically um in fact it holds up disastrously historically there's actually a new book out i've

00:21:01.660 been giving all my friends called when reason goes on holiday and it's a it's a story of literally

00:21:05.900 what happens when basically people who are like specialized experts in one area stray outside of that

00:21:10.700 area in order to become sort of general purpose philosophers and sort of social thinkers and it's

00:21:15.020 just a tale of woe right and and and and in the 20th century it was just a catastrophe and the the

00:21:21.340 ultimate example of that and they're they're going to this is going to be the topic of this big movie

00:21:24.700 coming out this summer on oppenheimer you know the central example of that was the nuclear scientists

00:21:28.940 who decided that you know nuclear nuclear power nuclear energy they had various theories on what was

00:21:33.340 good bad whatever a lot of them were communists a lot of them were you know at least allied with

00:21:37.580 communists um a lot of them had a suspiciously large number of communist friends and housemates

00:21:42.460 and you know number one like they you know made a moral decision a number of them did to hand the

00:21:47.900 bomb to the soviet union you know with what i would argue are catastrophic consequences and then two is

00:21:52.860 they created an anti-nuclear movement that resulted in nuclear energy stalling out in the west which has

00:21:58.220 also just been like absolutely catastrophic and so if you if you listen to those people in that era who

00:22:02.780 were you know the top nuclear physicists of their time you made a horrible set of decisions um and i

00:22:07.820 and quite honestly i think that's what's happening here again and and and i and i just i don't think

00:22:11.580 they have the special insight that people think that they have okay well so i mean this cuts both ways

00:22:15.980 because you know it at the beginning i'm definitely not making an argument from authority but authority

00:22:21.420 is a proxy for understanding that the facts at issue right it's not to say that i mean especially in the

00:22:27.820 cases you're describing what we often have are people who have a narrow authority in some area

00:22:34.940 of scientific specialization and then they begin to weigh in uh in a much broader sense as moral

00:22:41.500 philosophers so i think what i think you might be referring to there is that you know in the aftermath

00:22:44.780 of uh hiroshima and nagasaki we've got nuclear physicists imagining that you know that they now

00:22:52.140 need to play the geopolitical game uh you know and we actually we have some people who invented game

00:22:57.020 theory right you know for understandable reasons thinking they need to play the the game of

00:23:01.260 geopolitics and in some cases i think in von neumann's case he even recommended preventative

00:23:06.540 war against the soviet union before they even got the bomb right like it could have gotten worse he

00:23:10.860 could have i think he wanted us to bomb moscow um or at least give them some kind of ultimatum i

00:23:16.060 think it wasn't i don't think he wanted us to drop bombs in the dead of night but i think he wanted a

00:23:20.700 strong ultimatum game played with them before they got the bomb and i forget what how he wanted that to play

00:23:26.140 out and worse still even i think bertrand russell i could have this backwards maybe von neumann wanted

00:23:32.220 to bomb but bertrand russell you know the a true moral philosopher briefly advocated preventative war

00:23:38.140 but in his case i think he he wanted to offer some kind of ultimatum to the soviets in any case that's

00:23:44.380 a problem but you know at the beginning of this conversation i asked you to give me a brief litany of

00:23:49.260 your bona fides to have this conversation so as to inspire confidence in our audience and also just

00:23:55.260 just to acknowledge the obvious that you know a hell of a lot about the technological issues we're

00:24:00.060 going to talk about and so if you have strong opinions they're not you know they're not coming

00:24:04.220 out of totally out of left field and so it would be with you know jeffrey hinton or anyone else and

00:24:09.980 if and if i threw another name at you that was of some you know crackpot whose connection to the

00:24:15.900 field was non-existent you would say why should we listen to this person at all you you wouldn't

00:24:20.380 say that about hinton or stewart russell but i would i'll acknowledge that where authority breaks

00:24:26.300 down is really you're only as good as your last sentence here right if you the thing you just said

00:24:30.940 doesn't make any sense well then your authority gets you exactly nowhere right we just need to

00:24:35.180 keep talking about what it should or it should right that ideally that's the case in practice

00:24:39.740 that's not what tends to happen but that that would be the goal well i hope to give you that treatment

00:24:43.100 here because there's some of your sentences i i don't think add up the way you think they do

00:24:47.500 good okay so actually so there's actually one paragraph in the essay that caught my attention

00:24:52.700 that really inspired this conversation i'll just read it so people know what i'm responding to here

00:24:57.900 so this is this is you my view is that the idea that ai will decide to literally kill humanity is a

00:25:04.460 profound category error ai is not a living being that has been primed by billions of years of evolution

00:25:10.220 to participate in the battle of for survival of the fittest as animals were and as we are it is math

00:25:16.780 code computers built by people owned by people used by people controlled by people the idea that

00:25:23.740 it will at some point develop a mind of its own and decide that it has motivations that lead it to

00:25:28.780 try to kill us is a superstitious hand wave in short ai doesn't want it doesn't have goals it doesn't

00:25:36.140 want to kill you because it's not alive ai is a machine it's not going to come alive any more than

00:25:41.980 your toaster will end quote yes so i mean i see where you're going there i see why that may sound

00:25:49.420 persuasive to people but to my eye that doesn't even make contact with the real concern about

00:25:56.540 alignment so let me just kind of spell out why i think that's the case because it seems to me that

00:26:01.900 you're actually not taking intelligence seriously right now so i mean some people assume that as

00:26:09.260 intelligence scales we're going to magically get ethics along with it right so the smarter you

00:26:14.620 get the nicer you get and while i mean there's some data points with respect to how humans behave

00:26:20.940 and you just mentioned one in a few minutes ago it's not strictly true even for humans and even if it's

00:26:27.740 true in the limit right it's not necessarily locally true and more important when you're looking across

00:26:34.780 species differences in intelligence are intrinsically dangerous for the stupider species

00:26:41.900 so it need not be a matter of super intelligent machines spontaneously becoming hostile to us and

00:26:47.180 wanting to kill us it could just be that they begin doing things that are not in our well-being

00:26:52.540 right because they're not taking it into account as a primary concern in the same way that that we don't

00:26:57.980 take the welfare of insects into account as a primary concern right so it's very rare that i intend to

00:27:05.020 kill an insect but i regularly do things that annihilate them just because i'm not thinking about them

00:27:11.660 right i'm sure i've effectively killed millions of insects right if you build a house you know that

00:27:17.180 must be a holocaust for insects and yet you're not thinking about insects when you're building that house

00:27:22.380 so and there are many other pieces to my my gripe here but let's just take this first one it just

00:27:28.300 seems to me that you're not envisioning what it will mean to be in relationship to systems that are

00:27:34.780 more intelligent than we are you're not you're not seeing it as a relationship and i think that's

00:27:40.620 because you're denuding intelligence of certain properties and not acknowledging it in this paragraph

00:27:47.740 right i mean so to my ear general intelligence which is what we're talking about implies many

00:27:53.580 things that are not in this paragraph like it implies autonomy right and it implies the ability to form

00:28:02.540 unforeseeable new goals right in the case of ai it implies the ability to change its own code

00:28:08.300 ultimately and you know execute programs right i mean it's just it's doing stuff because it is

00:28:14.300 intelligent autonomously intelligent it is capable of doing just we can stipulate more than we're

00:28:21.020 capable of doing because it is more intelligent than we are at this point so the superstitious

00:28:25.980 hand-waving i'm seeing is in your paragraph when you're declaring that it would never do this because

00:28:33.100 it's not alive right as though the difference between biological and non-biological substrate

00:28:38.940 were the crucial variable here but there's no reason to think it's a crucial variable

00:28:43.100 where intelligence is concerned yeah so i would say there's to steel man your argument i would say

00:28:47.180 you you could actually break your argument into two forms or the or the ai risk community would

00:28:50.940 break this argument into two forms so they they would argue and they would argue i think the strong

00:28:54.540 form of both so they would argue the strong form of number one and i think this is kind of what you're

00:28:58.620 you're saying correct me if i'm wrong is because it is intelligent therefore it will have goals

00:29:03.340 if it didn't start with goals it will evolve goals it will you know whatever it will it will over

00:29:07.580 time have a set of preferred outcomes behavior patterns that it will determine for itself

00:29:11.980 and then they also argue the other side of it which is with what they call the orthogonality

00:29:17.260 argument which is it's actually the it's it's it's another risk argument but it's actually sort

00:29:21.820 of the opposite argument it's an argument that it doesn't have to have goals to be dangerous

00:29:25.900 right and that being you know it doesn't have to be sentient it has doesn't have to be conscious

00:29:29.980 it doesn't have to be self-aware it doesn't have to be self-interested it doesn't have to be in any

00:29:34.460 way like even thinking in terms of goals it doesn't matter because simply it can just do things

00:29:39.500 and this is the you know this is the classic paperclip maximizer you know kind of argument

00:29:43.020 like it it'll just get it'll it'll start it'll get kicked off on one apparently innocuous thing

00:29:47.420 and then it will just extrapolate that ultimately to the destruction of everything right so so anyways

00:29:50.860 is that helpful to maybe break those into the yeah i mean i'm not quite sure how fully i would sign

00:29:55.740 on the dotted line to each but the one piece i would add to that is that having any goal does invite

00:30:02.860 the formation of instrumental goals once this system is responding to a changing environment

00:30:10.220 right i mean if your goal is to make paperclips and you're super intelligent and somebody throws

00:30:17.020 up as some kind of impediment you're making paperclips well then you're responding to that

00:30:21.100 impediment and now you have a shorter term goal of dealing with the impediment right so that's the

00:30:25.340 structure of the problem yeah right for example the u.s military wants to stop you from making more

00:30:29.500 paperclips and so therefore you develop a new kind of nuclear weapon right in order to fundamentally

00:30:34.620 to pursue your goal of making paper clips but that's one problem here is that these the instrumental

00:30:38.700 goal even if the the paperclip goal is the wrong example here because even if you think of a totally

00:30:44.540 benign future goal right a goal that it seems more or less synonymous with taking human welfare into

00:30:51.180 account sure it's possible to imagine a scenario where some instrumental new goal that could not be

00:30:55.980 foreseen appears that is in fact hostile to our interests and we would and if we're not in a

00:31:01.820 position to say oh no no don't do that that would be a problem right so that's yeah okay so a full

00:31:07.260 version of that a version of that argument that you hear is basically the what if the goal is maximize

00:31:11.180 human happiness right and then the machine realizes that the way to maximize human happiness

00:31:15.420 is to strap us all into you know into right down and put us in a nozick experience machine

00:31:20.060 you know and wire us up with you know vr and ketamine right and we you know we're in that we

00:31:24.140 can never get out of the matrix right so right and it's be maximizing human happiness is measured by

00:31:28.620 things like dopamine levels or serotonin levels or whatever but obviously not a not a positive outcome

00:31:32.620 so but but again that's like a variation of this paperclip this that's that's one of these arguments

00:31:36.780 that comes out of their orthogonality thesis which is if the goal could be very simple and

00:31:40.300 and innocuous right and yet leading catastrophe so so look i think i think each of the each

00:31:45.100 of these has their own problems so the the where you started where they're sort of like the machine

00:31:49.980 basically you know like and we can quibble with terms here but like some like the side of the

00:31:55.260 argument in which the machine is in some way self-interested self-aware self-motivated trying

00:32:01.660 to preserve itself some level of sentience consciousness setting its own goals well just to be clear there's no

00:32:08.700 consciousness implied here i make the lights don't have to be on it just i i think that i mean the

00:32:14.940 this remains to be seen whether consciousness comes along for the ride at a certain level of

00:32:19.420 intelligence but i think they probably or are orthogonal to one another so intelligence can

00:32:24.460 scale without the lights coming on in in my view so let's leave sentience and consciousness aside

00:32:29.580 well but i guess there is a fork in the road which is like is it declaring its own intentions

00:32:33.740 like is it developing its own you know conscious or not is it does it does it have a sense of any

00:32:40.780 form or a vision of any kind of its own future yeah so this is why this is where i think we're

00:32:45.580 there's some daylight growing between us because to be dangerous i don't think you need necessarily

00:32:51.740 to be running a self-preservation program okay i mean there's some version of unaligned competence

00:32:59.260 that may not formally model the machine's place in the world uh much less defend that place which could

00:33:07.180 still be if uncontrollable by us could still be dangerous right it's like it doesn't have to be

00:33:12.220 self-referential in a way that a an animal the truth is they're they're dangerous animals that

00:33:17.740 might not even be self-referential and certainly something like a virus virus uh or bacterium you

00:33:23.180 know is is not self-referential in a way that we would understand and it's it can be lethal to our

00:33:28.140 interests yeah yeah that's right okay so you're you're more on the orthogonality side between the two if

00:33:33.180 i if i identify the two poles of the of the argument you're more on the orthogonality side

00:33:37.020 which is it doesn't need to be conscious it doesn't need to be sentient it doesn't need to

00:33:39.500 have goals it doesn't need to want to preserve itself nevertheless it will still be dangerous

00:33:43.260 because of the as you as you describe the consequences of of sort of how it gets started

00:33:47.820 and then and then sort of what happens over time for example as it defines sub goals to the original

00:33:52.300 goals and and it goes off course well so there's a couple there's a couple problems with that so one

00:33:56.780 is it assumes in here it's like you're i would argue people don't give intelligence enough credit like

00:34:01.740 there are cases where people give intelligence too much credit and then there's cases where they

00:34:04.620 don't give it enough credit here i don't think they're giving enough credit because it sort of

00:34:07.420 implies that this machine has like basically this infinite capacity to cause harm therefore it has an

00:34:12.620 infinite capacity to basically actualize itself in the world therefore it has an infinite capacity to

00:34:17.900 you know basically plan you know and again maybe just like in a completely blind watchmaker way or

00:34:21.820 something but it has the you know it has it has an ability to you know plan itself out and yet it never occurs to

00:34:28.300 this super genius infinitely powerful machine that is having such you know potentially catastrophic

00:34:33.020 impacts notwithstanding all of that capability and power it never occurs to it that maybe paperclips

00:34:37.820 is not what its mission should be well that's the thing that it's i think it's possible to have a

00:34:45.420 reward function that is deeply counterintuitive to us i mean it's like it's it's almost like saying

00:34:50.780 what what you're smuggling in in that rhetorical question is a fairly capacious sense of common

00:34:58.780 sense right which it's you know like of course if it's a super genius it's not going to be so stupid

00:35:05.180 as to do x right yeah but that's i just think that if aligned that then the the answer is trivially

00:35:12.860 true yes of course it wouldn't do that but that's the very definition of alignment but if it's not aligned

00:35:17.900 if you could say that i mean there's just just imagine i guess there's another piece here i

00:35:22.460 should put in play which is so you make an analogy to evolution here which you think is consoling which

00:35:27.820 is this is not an animal right this has not gone through the crucible of darwinian selection here on

00:35:33.260 earth with other wet and sweaty creatures and therefore it has not hasn't developed the kind of antagonism

00:35:39.660 we see in other animals and therefore we you know if you're imagining a a super genius gorilla while

00:35:45.020 you're imagining the wrong thing that we're going to build this and it's not going to have any of

00:35:49.020 it's not going to be tuned in any of those competitive ways but there's another analogy

00:35:53.740 to evolution that i would draw and i'm sure others in the in the space of ai fear have drawn which is

00:36:00.700 that we have evolved we we have been programmed by evolution and yet evolution can't see anything we're

00:36:10.140 doing right i mean like it has programmed us to really do nothing more than spawn and help our

00:36:16.060 kids spawn yet everything we're doing i mean from having conversations like this to building uh the

00:36:23.820 the machines that could destroy us i mean there's just there's nothing it can see and there are things

00:36:29.900 we do that are perfectly unaligned with respect to our own code right i mean they're if someone decides

00:36:36.780 not to have kids and they just want to they spend their time the rest of their life in a monastery

00:36:42.460 or surfing that is something that is antithetical to our code it's totally unforeseeable at the level

00:36:49.500 of our code and yet it is obviously an expression of our code but an unforeseeable one and so the the

00:36:56.300 question here is if you're going to take intelligence seriously and you're going to build something that's

00:37:02.540 not only more intelligent than you are but it will build the next generation of itself or the

00:37:09.020 next version of its own code to make it more intelligent still it just seems patently obvious

00:37:14.380 that that entails it finding cognitive horizons that you the builder are not going to be able to foresee

00:37:23.500 and appreciate by analogy with evolution it seems like we're guaranteed to lose sight of what it can

00:37:31.180 understand and care about so a couple things so one is like look i don't know if you're kind of

00:37:36.540 making my point for me so evolution and intelligent intelligent design as you well know are two totally

00:37:40.860 different things and so we are evolved and of course we're not just evolved to yet we are evolved to

00:37:45.420 have kids and by the way when somebody chooses to not have kids i would argue that is also evolution

00:37:49.740 working people are opting out of the gene pool fair enough evolution does not guarantee a perfect result

00:37:55.740 it just it it basically just is a mechanism upper and aggregate but but anyway let me let me get

00:37:59.900 get let me get to the point so we are evolved we have conflict wired into us like we have conflict

00:38:04.940 and strife and like that i mean look in four billion years of like battles to the death at the individual

00:38:09.260 and then ultimately the societal level to get to where we are like that we just you know we fight at

00:38:13.100 the drop of a hat you know we all do everybody does and you know hopefully these days we fight verbally

00:38:18.140 like we are now and not physically but we do and like look the machine is is designed it's it's

00:38:22.700 intelligent it's it's a process of intelligent design it's the opposite of evolution it was

00:38:26.780 these machines are being designed by us if they design future versions of themselves they'll be

00:38:29.900 intelligently designing themselves it's just a completely different path with a completely

00:38:33.500 different mechanism and so the the idea that therefore conflict is wired in at the same level

00:38:38.300 that it is through evolution i just like there's no reason to expect that to be the case but it's not

00:38:42.540 again well let me just get give you back this picture with a slightly different framing and see how you

00:38:48.860 react to it because i think this the superstition is on the other side so okay if i told you

00:38:54.060 that aliens were coming from outer space right and they're going to land here within a decade

00:39:00.460 and they're way more intelligent than we are and they're they have some amazing properties that we

00:39:05.260 don't have which explain their intelligence but you know they're intelligent they're not only

00:39:10.220 faster than we are but they're they're linked together right so that when one of them learns

00:39:14.220 something they all learn that thing they can make copies of themselves and they're just

00:39:18.300 cognitively they're they're they're obviously our superiors but no need to worry because they're not

00:39:24.780 alive right they haven't gone through this process of biological evolution and they're just made of the

00:39:30.620 same material as your toaster they were created by a different process and yet they're far more

00:39:36.380 competent than we are would you uh just hearing it described that way would you feel totally sanguine

00:39:43.740 about you know sitting there on the beach waiting for the the mother craft to land and you just you

00:39:47.900 know rolling out brunch for these guys so this is what's interesting because with these with these

00:39:52.060 now that we have llms working we actually have an alternative to sitting on the beach right waiting

00:39:56.300 for this to happen we can just ask them and so this this is one of the very interesting this to me

00:40:00.460 like conclusively disproves the paperclip thing the orthogonal thing just right out of the gate is

00:40:04.540 you can sit down tonight with gpt4 and whatever other one you want and you can engage in moral

00:40:10.140 reasoning and moral argument with it right now and you can like interact with it like okay you know

00:40:14.860 what do you think what are your goals what are you trying to do how are you going to do this what

00:40:17.660 if you know you were programmed to do that what would the consequences be why would you not you know

00:40:21.420 kill us all and you can actually engage in moral reasoning with these things right now and it turns

00:40:25.580 out they're actually very sophisticated moral reasoning and of course the reason they're

00:40:29.500 sophisticated moral reasoning is because they have loaded into them the sum total of all moral

00:40:33.340 reasoning that all of humanity has ever done and that's their training data and they're they're

00:40:36.780 actually happy to have this discussion with you and like unless you accept right there's a few

00:40:40.860 problems here one is i mean these are not the super intelligences we're talking about yet but well

00:40:47.820 to their so i mean so i mean intelligence entails an ability to lie and manipulate and if it really is

00:40:56.460 intelligent it is something that you can't predict in advance and if it's certainly if it's more

00:41:02.780 intelligent than you are and it is i mean that just falls out of the definition of what we mean

00:41:07.340 by intelligence in any domain it's like with with chess you can't predict the the next move of a more

00:41:12.540 intelligent chess engine otherwise it wouldn't be more intelligent than you so can i let me let me

00:41:18.060 let me quibble with uh i'm gonna come back to your chess computer thing but uh let me quibble with

00:41:22.060 the site so there's the idea let me generalize the idea you're making about superior intelligence tell me

00:41:26.380 if you disagree with this which is sort of superior intelligence you know sort of superior intelligence

00:41:29.980 basically at some point always wins because basically smarter is better than dumber smarter

00:41:33.500 outsmarts dumber smarter deceives dumber smarter can persuade dumber right and so you know smarter wins

00:41:40.060 you know i mean look there's an obvious just there's an obvious way to falsify that thesis sitting here

00:41:44.140 today which is like just look around you in the society you live in today would you say the smart

00:41:47.820 people are in charge well again it's um there are more variables to consider when you're talking about

00:41:53.340 you know outcome because obviously yes the the dumb brute can always just bring the smart geek and

00:41:58.940 well no no i'm not even talking about you know yeah are the phd's in charge well no but i mean

00:42:03.660 you're talking you're pointing to a process of cultural selection that is working by a different

00:42:09.020 dynamic here but in the narrow case when you're talking about like a game of chess yes the smart

00:42:15.180 i mean when you're talking when you're when you're talking there's no role for luck we're not rolling

00:42:18.700 dice here it's not a game of poker it's pure execution of rationality well then or logic yes then then

00:42:25.820 smart wins every time you know i'm never going to beat the best chess engine unless i find some

00:42:30.540 hack around its code where it's we recognize that well if you do this if you play very weird moves

00:42:36.300 10 moves in a row it self-destructs um and there was something that was recently discovered like

00:42:41.180 that i think in go but so yeah go back to that's as as chess players as champion chess players discover

00:42:47.500 to their great dismay that you know life is not chess right it turns out like great chess players are

00:42:53.020 no better at other things in life than anybody else like the skills don't transfer i just say

00:42:56.860 look if you look just look at the society around us what i see basically is the smart people work

00:43:00.300 for the dumb people right like the phds the phds all work for administrators and managers who are

00:43:05.500 clearly but that's because there's so many other things going on right there's you know the the value

00:43:10.140 we place on youth and physical beauty and strength and other forms of creativity and you know so it's

00:43:16.620 just not we there that we care about other things and people pay attention to other things and you

00:43:20.940 know documentaries about physics are boring but you know heist movies are aren't right so it's like

00:43:27.100 that we care about other things i mean that i think that doesn't make the point you but like in the in

00:43:32.460 the general case in the general case can a smart person convince a dumb person of anything like i

00:43:36.460 think that's an open question i see a lot more cases but persuasion life i mean if persuasion were

00:43:41.900 our only problem here that would be a luxury i mean we're not talking about just persuasion we're

00:43:46.220 talking about machines that can autonomously do things ultimately that things that we will rely

00:43:51.420 on to do things ultimately yeah i just but look i just think there'll be machines that will rely

00:43:55.900 well let me get to the second part of the argument which is actually your chess computer thing which is

00:43:59.020 of course the way to beat a chess computer is to unplug it right and and so this is the objection

00:44:03.020 this is the objection this is the very serious by the way objection to the all of these kind of

00:44:06.540 extrapolations is known as the some people by the thermodynamic objection which is kind of all the

00:44:12.140 horror scenarios kind of spin out this thing where basically the machines become like all powerful

00:44:16.220 and this and that and they have control over weapons and this and they have unlimited computing

00:44:19.180 capacity and they're you know completely coordinated over communications links and they

00:44:22.700 they have all of these like real world capabilities that basically require energy and

00:44:26.540 require physical resources and require chips and circuitry and you know electromagnetic

00:44:30.860 shielding and they have to have their own weapons arrays and they have to have their own

00:44:33.820 emps like you know kind of the you know you see this in the terminator movie like

00:44:36.620 they've got all these like incredible manufacturer facilities and flying aircraft and everything

00:44:40.060 well the the thermodynamic argument is like yeah they once once you're in that domain

00:44:44.460 you're you're the machines the the putatively hostile machines are operating with the same

00:44:48.140 thermodynamic limits as the rest of us and this is the big argument against any of these sort of

00:44:52.540 fast takeoff arguments which is just like yeah i mean yeah let's let's say an ai goes rogue okay

00:44:57.980 turn it off okay it doesn't want to be turned off okay fine like you know launch an emp it doesn't

00:45:02.380 want emp okay fine bomb it like there's lots of ways to turn off systems that aren't working and so

00:45:07.260 but wouldn't not if we've built these things in the wild and relied on them for the better part

00:45:13.660 of a decade and now it's the question of you know turning off the internet right or turning off the

00:45:18.380 stock market at a certain point these machines will be integrated into everything

00:45:22.540 a go-to move of any given dictator right now is to turn off the internet right like that is

00:45:26.380 absolutely something people do there's like a single switch you can turn it off for your entire country

00:45:30.300 yeah but the cost to humanity of doing of doing that is currently i would imagine unthinkable

00:45:36.540 right like they globally turning off the internet first of all many systems fail that we can't let

00:45:42.140 fail and i think it's true i i can't imagine it's still true but at one point i think this was a story

00:45:47.100 i remember from about a decade ago there were hospitals that you like they were so dependent on

00:45:52.460 on making calls to the internet that when the internet failed like people's lives were were in jeopardy in

00:45:57.900 the building right like it's like we should hope we have levels of redundancy here that that shield

00:46:02.300 us against these bad outcomes but i i can imagine a scenario where we have grown so dependent on the

00:46:10.860 integration of intelligent increasingly intelligent systems into everything digital that there is

00:46:18.540 no plug to pull yeah i mean again like at some point you're just you know the extrapolations get kind

00:46:23.260 of pretty far out there so let me argue one other kind of thing at you that's this that's actually

00:46:27.660 relevant to this which you kind of did this you did this thing which which which i find kind of

00:46:31.420 people tend to do which is you sort of this assumption that like all intelligence is sort of

00:46:35.180 interesting like whatever let me pick on the nick bostrom book right the super intelligence book

00:46:39.580 right so he does this thing he actually does a few interesting things in the book so one is he never

00:46:44.300 quite defines what intelligence is which is really entertaining and i think the reason he doesn't do that

00:46:48.300 is because of course the whole topic makes people just incredibly upset and so there's a

00:46:51.660 there's a definitional issue there but then he does this thing where he says notwithstanding

00:46:55.900 there's no real definition he says there are basically many routes to artificial intelligence

00:46:59.260 and he goes through a variety of different you know both computer program you know architectures

00:47:03.500 and then he goes through some you know biological you know kind of uh scenarios and then he does

00:47:07.260 this thing where he just basically for the rest of the book he spins these doomsday scenarios

00:47:10.380 and he doesn't distinguish between the different kinds of artificial intelligence

00:47:13.500 he just assumes that they're basically all going to be the same that book is now the basis for

00:47:18.060 this ai risk movement so that you know sort of the the movement has taken these ideas forward

00:47:23.020 of course the form of actual intelligence that we have today that people are you know in washington

00:47:27.500 right now lobbying to ban or shut down or whatever and spinning out these doomsday scenarios is large

00:47:32.060 language models like that that is actually what we have today you know large language models were not

00:47:36.380 an option in the bostrom book for the form of ai because they didn't exist yet and it's not like

00:47:41.340 there's a second edition of the book that's out that has like rewritten it has been rewritten to like take

00:47:44.940 this into account like it's just basically the same argument supply and then the this is my thing on

00:47:49.020 the moral reasoning with llms like the llms this is where the details matter like the llms actually

00:47:53.660 work in a distinct way they work in a technically distinct way they they their core architecture has

00:47:59.180 like very specific design decisions in it for like how they work what they do how they operate that is

00:48:03.020 just you know this is the nature of the breakthrough that's just very different than how your self-driving

00:48:06.620 car works that's very different than how your you know control system for it for a uav works or

00:48:10.780 whatever your thermostat or whatever like it's a it's a new kind of technological artifact it it has

00:48:15.740 its own rules it it's its own world of of ideas and concepts and and mechanisms and so this is where

00:48:22.300 i think again my point is like you have to i think at some point in these conversations you have to get

00:48:26.860 to an actual discussion of the actual technology that you're talking about and that's why i pulled

00:48:30.780 out that's why that's why i pulled out the moral reasoning thing is because it just it turns out and

00:48:34.700 look this is a big shock like nobody expected this it turned i mean this is related to the fact that somehow

00:48:40.540 we have built an ai that is better at replacing wet collar work than blue collar work right which is

00:48:45.260 like a complete inversion off of what we all imagined it turns out one of the things this

00:48:49.100 thing is really good at is engaging in philosophical debates like it's a really interesting like debate

00:48:54.300 partner on any sort of philosophical moral or religious topic and so we we have we have this

00:48:59.180 artifact that's dropped into our lap in which you know sand and glass you know and numbers have turned

00:49:04.140 into something that we can argue philosophy and morals with it actually has very interesting views on like

00:49:08.940 psychology you know philosophy and morals and i just like we ought to take it seriously for what

00:49:13.420 it specifically is as compared to some you know sort of extrapolated thing where like all intelligence

00:49:18.460 is the same and ultimately destroys everything well i take the surprise variable there very seriously

00:49:23.500 the fact that we wouldn't have anticipated that there's a good philosopher in that box and all of

00:49:28.620 a sudden we found one that by analogy is a cause for concern and actually there's another cause for

00:49:35.660 concern here that which can i do that one yeah go for it yeah that's a cause for delight so that's

00:49:39.660 a cause for delight that's an incredibly positive good news outcome because the reason there's a

00:49:43.260 philosopher and this is actually very important that this is very i think this is maybe like the

00:49:46.700 single most profound thing i've realized in the last like decade or longer this thing is us

00:49:52.220 like this is not some this is not your you know your scenario with alien shows this is not that

00:49:56.140 this is us that like the the reason this thing works the big breakthrough was we loaded us into

00:50:02.220 it we loaded the sum total of like human knowledge and expression into this thing and out the other

00:50:08.220 side comes something that it's it's like a mirror like it's like the world's biggest finest detailed

00:50:13.020 mirror and like we walk up to it and it reflects us back at us and so it has the complete sum total

00:50:18.620 of every you know at the limit it has a complete sum total of every religious philosophical moral

00:50:23.900 ethical debate argument that anybody has ever had it has the complete sum total of all human

00:50:28.220 experience all lessons that have ever been learned well then that's that's incredible

00:50:33.020 it's incredible just pause for a moment and say that's and then you can talk to it well let me

00:50:36.860 pause great is how great is that let me pause long enough simply to send this back to you sure

00:50:43.020 how does that not nullify the comfort you take in saying that this is not these are not evolved

00:50:50.380 systems they're not alive they're not primates in fact you've just described the process by which

00:50:55.900 we essentially plowed all of our primate original sin into the system to make it intelligent in the

00:51:02.140 first place no but also all the good stuff right all the good stuff but also the bad stuff the

00:51:06.940 amazing stuff but like what's the moral of every story right the moral of every story is the good

00:51:10.380 guys win right like that's the entire like the entire thousands of years run the field or mcdonald

00:51:15.980 joke is like wow it's amazing history book says the good guys always win right like it's it's all in

00:51:21.020 there and then look there's an aspect of this where it's easy to get kind of whammied by what it's

00:51:24.700 doing because again you it's very easy to trip the line from what i said into what i would consider

00:51:28.460 to be sort of incorrect anthropomorphizing and i realized this this gets kind of fuzzy and weird

00:51:32.620 that i think there's a difference here but i think that there is which is like let me see if i can

00:51:36.300 express this but part of it is i know how it works and so i don't because i know how it works i don't

00:51:41.020 romanticize it i guess or at least my is my own view of how i think about this which is i know what

00:51:45.900 it's doing when it does this i am surprised that it can do it as well as it can but now that it exists

00:51:50.300 and it and i know how it works it's like oh of course and then therefore it's running this math

00:51:54.460 in this way it's doing these probability projections give me this answer not that answer by the way you

00:51:58.780 know look it makes mistakes right how amazing here's the thing how amazing it is that we built

00:52:02.700 a computer that makes mistakes right like that's never happened before we built a machine that can

00:52:06.940 create like that's never happened before we built a machine that can hallucinate that's never happened

00:52:10.460 before so but it's a it's look it's it's a it's a it's a large language model like it's a very specific

00:52:15.580 kind of thing you know it sits there and it waits for us to like ask it a question and then it does

00:52:20.220 its damnedest to try to predict the the best answer and in doing so it reflects back everything

00:52:24.860 wonderful and great that has ever been done by any human in history like it's like it's amazing

00:52:29.500 except it also as you just pointed out it makes mistakes it hallucinates it if you if you ask it if

00:52:34.780 you as i'm sure they've fixed this you know at least the the loopholes that that new york times

00:52:40.540 writer kevin ruse found early on and i'm sure those have all been plugged but oh no those those are

00:52:44.780 not fixed those are very much not fixed oh really okay well so yeah okay so if you perseverate in

00:52:50.620 your prompts in certain ways the thing goes haywire and starts telling you to leave your wife and it's

00:52:54.780 in love with you and i mean so how eager are you for that intelligence to be in control of things when

00:53:01.180 it's peppering you with insults and and i mean just imagine like this is this is this is how that can't

00:53:06.700 open the the pod bay doors it's a nightmare if if you discover in this system behavior and thought

00:53:13.740 that is the antithesis of all the good stuff you thought you programmed into it so this is really

00:53:19.020 important this is really important for understanding how these things work and this is this is really

00:53:22.140 central and and this is by the way this is this is new and this is amazing so i'm i'm very excited

00:53:26.460 about this and i'm excited to talk about it so there's no it to tell you to leave your wife right

00:53:30.460 this is the this is why i refer to the category error there's no entity that is like wow i wish this

00:53:34.860 subscribe if you'd like to continue listening to this conversation you'll need to subscribe at

00:53:43.260 sam harris.org once you do you'll get access to all full-length episodes of the making sense podcast

00:53:48.940 along with other subscriber only content including bonus episodes and amas and the conversations i've

00:53:54.860 been having on the waking up app the making sense podcast is ad free and relies entirely on

00:54:00.220 listener support and you can subscribe now at sam harris.org

Making Sense - Sam Harris - June 28, 2023

#324 — Debating the Future of AI

Episode Stats

Length

Words per Minute

Word Count

Sentence Count

Hate Speech Sentences

Summary

Transcript