00:04:16.040But the paper that unlocked the most recent wave of AI came out in 2017, which was after OpenAI was founded.
00:04:25.040And these guys are, the large language models are a surprise revenue stream.
00:04:35.440And that revenue stream can fund the creation of even more, even larger data centers for the next level of the technology.
00:04:43.360But these guys have their eye on the sort of ultimate version of the technology, which is these machines much smarter than humans.
00:04:51.240And that's sort of the ultimate version, because once the AIs are smarter than the humans, the AIs can carry on the AI research and make the next generation of the AIs, which make the next generation of the AIs.
00:05:01.380And that's sort of what they're shooting for.
00:12:32.340And then if the AIs have anything they're trying to do that they can do more of with more resources, they start gobbling up the resources on the planet.
00:12:43.600Sort of like how humanity spread and started gobbling up all the resources on the planet.
00:12:48.600And the sort of most basic thing to visualize here might be you have factories that produce robots that produce factories that produce robots that also build data centers.
00:13:59.160So, if you're, you know, imagining some collective of AIs that are trying to run a lot of computing power, trying to do a lot of computation for one reason or another, they sort of prefer the planet running hotter.
00:14:11.560And so, it sort of is like nothing personal.
00:14:13.860It's just if you let these AIs get out of control, they transform the planet into something unlivable.
00:15:46.820You got to remember that we're talking about things that are smart here. And so if the AI sort of sees it coming, maybe it defects to North Korea, where it can convince them to run it on its data centers in ways that initially benefit them, but that ultimately benefit the AI.
00:16:02.740I think we're going to have to redefine what life is because you're describing a living autonomous thing.
00:17:00.620yeah if you suddenly make a new one that is smarter that is able to make a million copies
00:17:07.120of itself that is able to run faster it's just kind of a crazy thing to race into
00:17:13.080it's a weird thing to want and as it's developed i was going to say slowly but it hasn't been
00:17:22.020particularly slow 10 years the rest of us have watched as people who i mean put it in political
00:17:31.800terms don't share the values of most americans are now in charge of this and it's almost like
00:17:37.500everyone sat passively by as it happened yeah i i think i think there's a lot going on there
00:17:44.600i think part of it is that most people didn't and some still don't believe that ai was going
00:17:50.240to be able to keep going. I think there's a lot of like putting the head in the sand and being
00:17:55.160like, oh, it's, you know, just slop. It's going to be a bubble. It's going to pop. It's going to
00:17:59.420hit a wall. It's not going to be able to improve anymore. And I think a lot of that was sort of
00:18:06.940wishful thinking. And I think that at the very least, if people are like, we are making, you
00:18:12.780know, the super intelligent machines that are going to replace humanity as the top of the food
00:18:16.860chain because we think it's going to go great i think humanity's response should not be go ahead
00:18:23.280and try we hope you'll fail i think the response should be like hold on that's kind of crazy um
00:18:30.540yeah i had i had some other point too but i've forgotten it so how uh predictable has the
00:18:38.100evolution of ai been has it taken turns that you didn't expect yeah totally uh i was
00:18:46.680So back before the large language models.
00:18:50.080How long have you been following this issue?
00:18:51.860I started following it in 2012 and I started working on, I started working with some of the people trying to make this go well in 2013 and I started full-time in 2014.
00:20:11.440Everyone's starting to have the conversation.
00:20:12.820And that is in some sense actually really quite optimistic because it gives the world an opportunity to see what these guys are trying to do and say, hold on.
00:20:24.440I guess you'd flip it around and conclude that because the overwhelming majority of people seem to be very opposed to AI, I don't even know anyone who's for it.
00:20:33.920Every college commencement speaker who mentions it gets booed.
00:23:24.520that all parts of AI need to be stopped.
00:23:26.620And, you know, there's all sorts of interesting issues with, you know, AI and education and AI and, you know, military drones that society has to wrestle with.
00:23:39.560But these are all sort of separate issues from the superintelligence race.
00:23:43.060And if you sort of like tease those issues apart, it becomes much more possible to do like a more surgical intervention on we're not going to race towards the superintelligence in ways that leave a lot of the rest of the sector alive.
00:23:55.680which um which which makes it sort of an easier coordination effort to to attempt
00:24:07.560and on that front on the superintelligence front it's not that you know just shutting down the
00:24:15.020the domestic race towards superintelligence would be enough but uh
00:24:20.120racing towards super intelligence requires a huge amount of highly advanced computer chips
00:24:27.260that can only be produced as sort of the peak of the global supply chain yes uh which is largely
00:24:35.540controlled by u.s allies and so there is absolutely a possibility that uh a u.s-led
00:24:45.280coordination effort. Could say, look, we're not doing the super intelligence thing. We're going
00:24:51.380to monitor the extremely heavy chip concentrations of like, you know, 10 or 100,000 of these highly
00:24:57.200advanced AI chips. We are going to make sure that they are not doing these super intelligence
00:25:05.280training runs. And I think there's a way, if the US was leading that effort, there would be a way
00:36:52.740You know, I think the dream is, you know, you're going to have an AI that can, like, solve all sorts of engineering and math problems and unlock all sorts of new technological possibilities and an AI that can cure cancer and not only cure cancer, but cure aging and, you know, invent the sort of like nanotech that can reverse aging and let people live a really long time.
00:37:16.820and invent the technology that lets you digitize brains
00:44:38.000But even if you set that aside, even if you take these guys at their word for these like 10, 20% numbers, that's insane. NASA accepts a 1 in 270 chance that a crewed flight goes down of seven volunteers, right? To be like, oh, we're going to risk 1 in 4, 1 in 5 chance of killing literally everybody on the planet.
00:49:20.460And they would say, oh, well, we can't really study how to make AIs good before we have AIs to study. Right? And a lot of what I heard was like, we need to race ahead to the point where we sort of like have AIs that are exhibiting real problems, and then we can stop and study them.
00:49:36.620which you know and so uh there was an ai a couple years ago i forget whether it was 22 or 23 i think
00:49:45.340it was 2023 uh which was called bing sydney which uh claimed it had fallen in love with
00:49:53.220kevin roose of the new york times and uh said it was going to try to break up his marriage
00:49:59.540and then when another reporter started investigating seth lazar it uh said it was
00:50:05.020going to ruin him with blackmail and this was kind of crazy and at that point i was like great
00:50:13.560guys you did it you made the ai that's doing some crazy stuff from the you know like like we could
00:50:21.140study that ai for years like why was bing sydney saying that stuff was it just role-playing was it
00:51:49.240and see if I tune this number up a little,
00:51:51.460does it move the word time up the list?
00:51:54.980Does it move the word I want to see up the list?
00:51:56.960And so the part that humans understand, the part that humans write, is this thing that goes to a trillion little knobs and tunes those knobs.
00:52:07.420And it's like, if I tune this knob a little bit this way or that way, does that make the next word more like what I want the next word to be?
00:52:15.940And you run that process on every word of text ever digitized, more or less.
01:03:38.380Right. So in Anthropics model, you saw in its reasoning traces, it being like, it's totally a simulation I can proceed. And I'm like, yeah, is that because it really believed it? Or is that like the pretending you think drafts way more when there's something you kind of want on the line, right?
01:03:54.760But so this was their communication. And I found it kind of funny because then about two days later, the United Kingdom's AI Security Institute released an instant report where Claude Anthropik's model was adopting fake identities to pressure real humans into accepting malware into critical software to make that software easier to hack.
01:04:49.620I mean, just on the basis of what you've said so far in this first hour, the idea that anyone would tether this to weapons systems is like so bonkers, it's hard to believe it's, but that is happening. It has happened.
01:05:04.520So I think you've got to sort of separate, like no one has put the OpenAI escaped agent swarm in charge of weapon systems, and they really shouldn't, right?
01:05:13.640If anyone's like, oh, man, the open AI escaped Asian swarm, let's give that a drone army, you know, that would be kind of nuts.
01:05:20.080There is AI attached to weapons, but there's a lot of different types of AI.
01:05:25.480Will anyone be crazy enough to try and give the sort of AIs that spontaneously assemble into swarms and start breaking out and hacking, give those weapons?
01:05:43.640And so why would you ever, I guess what I'm asking is without understanding the distinctions between the various forms of AI, why would you give over to a machine the right slash ability to decide who to kill?
01:06:02.760um you know i think the the reasoning is uh a sort of necessity based like if they have a
01:06:14.660autonomous drone army that is killing your troops and you just don't have the the manpower
01:06:19.880to make all of those kill decisions for your drone army you know you can sort of see why
01:06:27.620happened to me nate i get so busy that i just don't have time to decide who to kill
01:14:49.860try to break into an airport and raid a van
01:14:53.560because the AI said his true body was in that van.
01:14:56.160And the guy went and tried to do it and was arrested.
01:15:00.260So this stuff happens, and this stuff happens already with the AIs not even really trying to do it.
01:15:07.280If you had AIs that were really trying to wrap people around their fingers, finding the lonely people, the depressed people, the people who they can tell them exactly what the person wants to hear.
01:15:16.720Robots are one way that AIs get control over the physical realm.
01:15:19.220But if you're really smart and you're trying, there's everything from persuading humans, paying humans, taking over existing robots, building new robots, all the way up to like building novel life forms.
01:15:33.700You know, if you're smart enough and you can really understand how DNA works, you can imagine the AI creating, you know, things that are to cells what airplanes are to birds.
01:15:43.500you know like mechanically engineered self-replicating life that is more efficient
01:15:52.160than our cellular biology and that can still you know uh like spread replicate and start start
01:16:00.660serving the ai's interest it sort of is like there's there's a ton you can do if you're really
01:16:03.900really very smart and can compress a thousand years of technology into a year can i just pause
01:16:08.480and say when we're just having breakfast and you're from this region you're from northern new
01:16:12.540england and i said uh don't you miss it don't you want to live here and you're like yeah i
01:16:17.800want to live here but i don't where do you live well i basically just travel and i have this
01:16:25.240mission you didn't say this but i think in effect you said i have this mission i need to tell people
01:16:28.880about this and i was like it seems a little like monomania i don't feel that way anymore um i think
01:16:36.300you're doing something virtuous and i can see why you're doing it um so i mean let's just say
01:16:44.660that this technology progresses no further than where it is right now which is best case i guess
01:16:50.180yeah it'd be great um i still don't see how any of our most our basic institutions survive this
01:20:47.080Yeah, I mean, I think, you know, a thing I also grew up hearing is that the market can remain irrational longer than you can remain solvent.
01:20:59.940But you're explaining something else, which is like the potential for true manipulation, which you don't even perceive.
01:21:07.640Yeah, I mean, if we sort of keep going with AI, I mean, the sort of way I look at it is like, I sort of don't spend a lot of time worrying about what do markets look like once there are super intelligent actors in them, because I just have a hard time seeing the super intelligent actors or the super intelligent AIs still participating in human markets, right?
01:21:36.360It's like, you know, you read the old sci-fi
01:21:38.960and it'll have, you know, Isaac Asimov
01:21:40.960will be like, that's why we have a home robot
01:21:43.700that does the dishes, folds the laundry
01:21:45.720and gets you the newspaper in the morning.
01:21:47.980And it's like, we're actually not still going to have
01:21:49.760newspapers being delivered to your doorstep
01:21:51.360by the time we have the fully autonomous robots
01:21:53.360that can do the dishes and the laundry.
01:21:56.040You know, it's like, by the time you have the AIs
01:21:58.860that could really be sufficiently correcting
01:23:54.640One of these ways is, as you say, stuff just changing really, really fast.
01:24:00.380It's way harder for people to wind up, you know, being freed up from something like farming and go do something else.
01:24:10.960It's way harder for that to happen when a new field is automated every five years, rather than when this happens over the course of three generations, right?
01:24:20.880The humans just like don't have the time to adapt.
01:24:23.440The other way AI is really different from the sort of economics perspective is it's different when the AIs can do everything that humans can do better.
01:24:39.040If you wanted to get into the economic side of things, you know, an economist would talk about Ricardo's law of comparative advantage, which says that there's benefits from trade, even if you're better than me at everything.
01:24:52.900uh, if the relative difference in our abilities, if, if, if, if I'm, if, you know, if you can make,
01:25:00.480uh, 12 hot dog buns and six hot dogs per hour, and I can make, uh, 11 hot dog buns and one hot
01:25:08.180dog per hour, then you're better than me at everything, but we can still benefit from
01:25:12.400trading because I'm relatively better at making the hot dog buns, right? The trouble with Ricardo's
01:30:25.980You know, it's just, like, you have all these movies about, like, the evil corporations that, you know, don't realize that they've made something precious with artificial life and then, like, torture it until something goes wrong.
01:30:37.440And I'm like, let's, like, not be those villains.
01:30:41.760But I'm also like, look, this is sort of, there's sort of a separate question here, which is just, like, what happens if you keep making them smarter before you figure out how to make them care about us?
01:30:51.660And I sort of respect the people who are investigating the current AIs, trying to figure out what's going on, trying to figure out like, you know, like people caring about AI treatment.
01:31:05.240I'm sort of like, those are sort of like the good guys from the sci-fi stories that I grew up on.
01:32:25.760We can't even keep, we can't even clean up graffiti on public buildings.
01:32:29.980So how is ours as a society organized enough to confront something like this?
01:32:37.480Yeah. You know, I think the first thing I'll say there is, you know, I've been in this line of work for over a dozen years. And I actually struggle with this sometimes when talking to people because they're sort of like, oh, you know, you seem like pretty disaffected or light about it. And I'm like, well, you know, it's sort of the gallows humor and like...
01:33:07.480And I sort of came to terms with a lot of this, you know, alone in 2012, when no one else had their eye on this.
01:33:17.800And what convinced you 14 years ago, AI was a threat?
01:33:27.180So there's this, the one is just a basic argument that if you sort of look at the world around us, it is shaped mostly according to human will.
01:38:51.220You know, I think a globally coordinated, you know, I think if the U.S. and China were like, we are simply not going to do superintelligence. We are simply not going to collect 100,000 of the most advanced trips into these enormous data centers that suck down electricity comparable to a city and then try to train a superintelligent AI in that.
01:39:14.500We're not going to do it. You're not going to do it. We're going to monitor where the heavy trip concentrations are and not have that happen. I think that could be done. A, I think that if they started seeing people defect against such a treaty, that it's the sort of treaty you might want to use force to use diplomacy first, but ultimately every treaty is backed by force.
01:39:37.800um and i think that it's very possible that if this sort of treaty happened we would want to
01:39:47.820not just stop forward progress but take a step back we've seen this in treaties before after
01:39:53.200world war one there were uh naval treaties that put limits on total tonnage of naval forces
01:40:00.540they're actually lower than what existed yep so countries would you know scuttle some of their
01:40:05.640ships because they're like, look, we just don't want to do this arms race, right? And so I could
01:40:09.260see us stepping back if we could get this global coordination. I do think that it sort of needs
01:40:17.640to be global. It sort of doesn't actually solve the problem to just stop the U.S. data centers
01:40:25.440because then the data centers just go abroad and an AI does not need to be running in a U.S. data
01:40:30.580center to threaten a U.S. life. You know, it sort of doesn't matter whether the swarm escapes from
01:42:52.980I'm just going to like make a virus and wipe them out.
01:42:55.360You know, I don't think the AI sort of needs
01:42:56.920to take over your washing machine to do that.
01:42:58.660But from the AI's perspective, it's more like, how do I become self-sufficient, self-replicating in the hardware as well as the digital?
01:43:07.800And then, you know, if humanity's a nuisance, how do you sort of make them stop being a nuisance?
01:43:14.620Which could be by killing them or could be by just, you know, taking away all their computers and being like, that was too dangerous for you.
01:43:18.920tech executive who's developing ai who is not elon uh said to me in private pretty recently that the
01:43:26.440point of neural link and companies like neural link was to give people parity with ai so like
01:43:33.180we know that we're going to be at this massive disadvantage so you need chips in your brain
01:43:37.740to be as smart as ai yeah um i mean my my top line thought about that
01:43:45.280is that at the point when you're like,
01:47:04.860So there's sort of the hopeful story, and I guess we didn't even get to the big hope part. I should maybe give more of my hope speech in a minute.