AI Will End Humanity. No One Knows How To Stop It - Dr Roman Yampolskiy
Episode Stats
Length
1 hour and 11 minutes
Words per minute
164.61461
Harmful content
Misogyny
2
sentences flagged
Toxicity
19
sentences flagged
Hate speech
17
sentences flagged
Summary
In this episode of Trigonometry, we talk to Dr. Lakshya Raman, a leading voice in the field of artificial general intelligence and AI safety. She is a professor of computer science at the University of California, Los Angeles, and the author of several books on the topic.
Transcript
00:00:05.000
It's capable of coming up with new weapons, new physics, new poisons.
00:00:13.000
It definitely has potential to lock in dictatorships.
00:00:22.000
What if humanity becomes sort of like, you know,
00:00:26.000
Like, you know, a nice pet for the AI to maintain, to look after.
00:00:33.660
Sometimes owners decide to put you to sleep, or neuter you.
00:00:49.380
I'm surprised that more people are not freaking out.
00:00:52.100
So, I guess the obvious question is, what do you advocate that we now do?
00:01:01.000
Trigonometry is proudly independent and sponsors like Hillsdale College make that possible.
00:01:06.040
Access their free library of world-class educational courses at hillsdale.edu slash trigger.
00:01:17.600
And you are one of the leading people in the AI safety world, I would say, both in terms
00:01:24.880
of the work you do, but also in terms of the things you say.
00:01:35.220
We are creating something with capacity to replace us or kill us.
00:01:41.120
And safety is what we're trying to do to prevent bad outcomes.
00:01:44.980
Everyone historically has been working on capabilities.
00:01:48.300
More capable systems replace human labor, replace creativity.
00:01:53.940
But very few people worked on how do we make sure it goes well.
00:01:57.220
There is no side effects, there is no abuse of this technology.
00:02:01.480
Now people are realizing, oh, there are military applications of this.
00:02:06.420
So we see the fight with Anthropic and Department of War.
00:02:10.520
But the bigger problem is if those systems go from narrow systems, subhuman, to human
00:02:22.580
All the things you've laid out, we've explored on the show before with different people,
00:02:28.900
But you say it with a level of confidence that tells me you have a sort of a vision
00:02:40.340
you're doing is you're asking me how i would destroy humanity and i have many good ideas
00:02:45.140
it's not what a super intelligent system would do it's capable of coming up with new weapons new
00:02:50.980
physics new poisons uh example i frequently use is squirrels versus humans it's a big cognitive gap
00:02:58.740
squirrels have no concept of how we can kill them all they don't know about guns they don't know
00:03:03.140
about traps it's outside of their world model likewise they cannot tell you how super intelligence
00:03:08.340
would specifically go about it. But there are many game theoretic reasons for why it's
00:03:12.520
a good idea not to have competing species, not to have humans create another super intelligence.
00:03:19.380
Maybe it just wants to do something with this environment and doesn't care about us.
00:03:24.640
But I guess the question would be, in terms of your certainty, why you believe that AI,
00:03:30.220
if it becomes artificial general intelligence, why it would hurt human beings? What would
00:03:36.560
the way that you think that would happen? So what I kind of started saying, it's not
00:03:40.800
because it hates you, because it wants to do something else and it doesn't care about you.
00:03:45.440
So maybe it wants to cool down the whole planet to improve how efficient compute is. Just it's more
00:03:52.400
capable of doing computation in a colder environment. So if it freezes the whole planet,
00:03:57.120
we die. Does it care about it? No, it doesn't matter. Maybe it wants to convert this planet
00:04:05.740
I'm giving kind of hypotheticals which are not grounded in anything, but the point is
00:04:11.020
it just doesn't have any built-in concern about your safety, your well-being.
00:04:16.560
If it wants to accomplish something and the side of it is humanity dies, it would not
00:04:22.720
Would we not be able to write the preservation of humanity into the basic code of what this
00:04:32.900
give them data all the data we have all of internet and then it learns something from the dark corners
00:04:39.460
of internet from libraries from stories and whatever it learns we're trying to figure out
00:04:44.900
we do experiments in those models we see what is it capable of what is it interested in but we study
00:04:51.700
it like we study biological artifacts you find a new species of animal on some island we're trying
00:04:57.540
to figure out what it's capable of. Does it have a poison? Does it have some interesting
00:05:03.860
social structure? That's what we're doing. We're not explicitly coding up those systems.
00:05:08.500
So no, nobody knows how to encode anything like that into the existing models. Nobody's claiming
00:05:14.340
to have a safety mechanism. Roman, you've been involved in this field for a long time.
00:05:20.820
When did you first start to get concerned about AI and the safety of AI?
00:05:26.260
So my PhD work was on safety of online casinos. And at the time, bots, poker bots, just started
00:05:34.980
to show up. And so the small concern we had about, are they going to collude and cheat the players?
00:05:42.200
Are they going to steal cyber infrastructure? So that was the initial kind of level of concern.
00:05:48.060
Obviously nothing like what we're talking about today. But as the bots got better and better,
00:05:53.000
Our ability to detect them, to prevent them, was not always keeping up.
00:05:58.540
And when we took it to extreme, to human level and beyond, there is no safety.
00:06:03.700
We simply don't know how to make sure their systems behave.
00:06:07.460
Because the worrying thing is, is what you're effectively saying is that we're creating technology
00:06:18.120
We don't have the imagination in order to see what happens in the long term with this technology.
00:06:23.000
we just create this technology and then it goes forth and multiplies quite literally in some cases
00:06:30.520
and the concern is if you look at social media social media started off as a way with mark
00:06:36.600
zuckerberg to compare girls on campus and yet here we are now nearly 20 years later and it's
00:06:41.480
completely unrecognizable from what it once was right so that's a great example it's unpredictable
00:06:47.320
how we will use technology, how it will impact everything.
00:06:50.820
So Facebook was meant to date pretty girls on campus
00:07:02.320
We're not creating technology in a traditional sense.
00:07:07.560
It doesn't take a malevolent human to abuse this technology.
00:07:17.320
Because, so if we use the example of Facebook, Facebook's mantra at the beginning was move fast and break stuff.
00:07:24.060
And because they wanted to take over and essentially they didn't care who got in their way.
00:07:28.800
They wanted to get to where they want to get to.
00:07:31.260
And when we met people from Silicon Valley, from the AI world, bear in mind we didn't meet the top people.
00:07:36.980
We just met a small portion of people and we talked to them.
00:07:39.600
I was concerned because it didn't seem to me that ethics and the long-term effects of this technology was forefront in their mind.
00:07:49.160
I'm not saying they were malevolent, I'm just saying it didn't appear that the long-term impact of this technology was their primary concern.
00:07:59.120
That's true. Historically, most people working in AI never took the time to think what happens if we succeed.
00:08:05.940
because it was so hard for so many years there was so little progress they had winters one after
00:08:11.540
another so they basically just worked on it tried to make as much progress as possible without ever
00:08:17.940
stopping and thinking well what if i am successful what if i create competing species something
00:08:23.140
smarter than humans is that good for us how will we interact with them and the last 10 years the
00:08:29.220
progress went exponential it went from basically we have no progress you have to hand code every
00:08:34.820
new application to those systems can scale they can learn they can transfer knowledge
00:08:39.620
and now it's hyper exponential because the ai itself is helping with research
00:08:44.020
but we haven't spent the time to decide do we want this do 8 billion people agree to this
00:08:51.380
experiment are they interested in having their jobs automated and that's just the economic
00:08:57.460
concerns not the safety concern well we'll talk about the economic concerns separately but i mean
00:09:02.660
One of the things that may seem particular to our audience, which is a not AI-specific audience,
00:09:08.420
the people who watch our show are just normal people going about their lives,
00:09:11.940
this may feel like we're talking about something in the distant future.
00:09:16.100
I was looking at the Calci odds for OpenAI getting AGI by 2030, and it's now over 52%,
00:09:24.580
and it's gone up 13 points this year so far. It seems to me like we're heading in the direction
00:09:32.020
of getting to AGI within what kind of time frame do you think? 2030 is somewhat conservative. Some
00:09:37.620
people are saying we already got there. We just haven't deployed it yet. However, I'm pretty sure
00:09:42.660
it could be a year or two. Wow. And so, you know, the big risk that you're talking about,
00:09:50.340
which is you create a super intelligence, you've basically created another species,
00:09:55.380
which is more powerful than you. And when we had Dworkesh Patel on the show, this is kind of like
00:10:00.820
Like I said to him, you've basically created this, like the Unsullied from Game of Thrones,
00:10:12.700
I don't know what that is, I have no idea, but sounds right.
0.99
00:10:18.340
The Unsullied were a group of slaves, slave warriors, who would obey every command, including
1.00
00:10:25.980
But I imagine, particularly given some of the things we've seen, maybe you'll correct
00:10:30.580
on this but i read about this experiment where they tell ai they're about to replace it and they
00:10:35.780
also give it some compromising information about the ceo and in some cases the ai will blackmail
00:10:41.460
the ceo to me that says it has a survival instinct already and anything that has a survival instinct
00:10:48.100
will necessarily put itself first is that fair so it wasn't the ceo that's one of the engineers but
00:10:54.020
it doesn't matter it does have self-preservation instinct and part of the reason it does is because
00:10:59.700
we kind of in a darwinian competition way select models which do they want to survive to the next
00:11:06.500
level the ones we delete or retrain they're not there to carry their intellectual payload so
00:11:14.660
that's exactly that they learn to detect that they are being tested and if they're being tested they
00:11:20.260
behave in a different way they want to pass the test they want to survive to deployment
00:11:25.380
that's exactly what we train them to do if a model fails a test we modify it we delete its memory we
00:11:33.540
replace it with another model so by definition of darwinian selection you'll get the ones which
00:11:39.720
pass the test the ones that deceive humans about their abilities and programming effect or lack
00:11:46.380
of abilities whatever it is we're trying to do to pass the test and it's already deceiving us
00:12:02.780
I'm surprised that more people are not freaking out.
00:12:05.780
I get people saying, oh, this is fear-mongering.
00:12:09.780
Most people don't understand what's about to happen.
00:12:21.780
yeah well that's not gonna happen is it because that doesn't look good because
0.99
00:12:27.000
the argument is if we don't do the Chinese world that's the dumbest
1.00
00:12:30.240
argument ever why so if I don't kill all my friends maybe someone else will kill
0.99
00:12:35.820
all my friends so I'll do it the argument is slightly less dumb than that
0.98
00:12:41.820
I think which is there is a gap between this thing becoming super intelligence
0.52
00:12:47.880
kills us all and we can't i mean the way you're explaining i think you know it's very persuasive
00:12:54.360
but some people will say it's not 100 let's say it's 99 even as as high as that in the interim
00:13:01.720
the technology will become a powerful weapon which our adversaries if they develop them first will
00:13:08.120
use to dominate us and to maybe even kill us whatever so we have to like nuclear weapons
00:13:14.360
develop our own AI deterrent. That's the argument. I don't think that's that dumb, is it?
00:13:19.880
So that argument makes sense, but it's super short term. It's while it's not human level,
0.97
00:13:25.400
while it's a tool below human level. So you have smarter drones, you're going to dominate
00:13:29.000
on a battlefield. Sure. But if you look at prediction markets, if you look at what leaders
00:13:34.520
of the labs are saying, we don't have that much room. The moment it flips general and then super
00:13:39.880
intelligent you have a weapon of mutually assured destruction it doesn't matter who creates super
00:13:44.920
intelligence if they don't control it it's the same outcome so some people argue better right
0.93
00:13:50.920
than dead you know chinese are building a pretty good country they haven't attacked us the best
0.79
00:13:57.000
business partners we have maybe we should take that risk another human species they are just like
0.60
00:14:04.760
us same preferences same values versus this alien species where we have no understanding and no
0.94
00:14:11.880
chance of competing but the chinese are not going to stop developing ai they have said that they
0.61
00:14:18.600
are very concerned about safety and if there was signal from us that we are not entering an arms
0.95
00:14:25.640
race they would really i suspect they would they are unlike our politicians not lawyers they are
00:14:32.600
scientists and engineers so there is a lot more understanding of what can happen here so you think
00:14:38.200
that it's possible that china and the united states could do some kind of deal to prevent
00:14:44.920
the development of super intelligence i think and you think that's the only way to save humanity i
00:14:49.480
could and should i think informally there is dialogue between american and chinese scientists
00:14:55.080
and they're very much in agreement on this issue and if chinese scientists are participating that
00:15:00.920
means it's approved by the chinese government they won't be able to do it independently so i think
00:15:06.280
we can do it at nation national level and i think at the corporate level i think daddy was on record
00:15:13.560
is saying if our slow down will pause as well so all we need is this external pressure to get them
00:15:20.120
together and all of them say okay this is dumb we're going to lose everything we are young rich
0.64
00:15:25.560
people. We can continue this. This is a pretty good deal. So why risk it all?
0.97
00:15:32.280
Roman, you said the words, people have no idea what's going to happen.
00:15:37.720
What is going to happen? So unpredictability is one of the problems with this technology.
00:15:46.200
I cannot tell you specifically what a smarter system will do. I can tell you general trends.
00:15:51.080
it will win a competition against me for playing chess it will outcompete me but what specific
00:15:57.000
moves is going to make i cannot tell you if i could i would be at that level so i cannot tell
00:16:02.680
you any specific things a super intelligence will do what i can tell you is we don't explain well
00:16:09.240
how it works we don't know how it works the explanations we get we don't fully comprehend
00:16:14.920
we cannot predict specific decisions and we cannot control them not in a direct sense giving orders
00:16:22.760
not in a delegated advisor sense because we lose all control if you're saying the system
00:16:28.200
is smarter than me it knows me better than i know myself why don't they just trust it to
00:16:33.000
make decisions for me well at that point you're not in control either it may make decisions you're
00:16:38.680
happy about maybe not we don't control it most people normal people think that people creating
00:16:46.520
this technology understand how it works and they can do things to ensure that it does good or bad
00:16:54.120
it doesn't do something that's not the case nobody explicitly programs them they are grown
00:17:00.840
from data and compute you get this alien plant and then you deal with it you study it you try
00:17:08.120
to understand what it does. At the same time, safety research stopped at the level of filters
00:17:14.720
and bans. So you have a list of topics not to talk about, a list of words not to say.
00:17:21.280
But it doesn't do anything to the model. It's after the fact filtering.
00:17:26.780
New year, new systems, right? This is the time when we all look at the messier parts of our
00:17:31.640
business and think there has to be a better way. And there is. Streamlining your communications
00:17:37.040
is one of the quickest and easiest upgrades you can make.
00:17:40.420
That's why today's episode is brought to you by Quo,
00:17:44.700
the smarter way to run your business communications.
00:17:49.520
Quo helps you and your team share one business number,
00:17:52.660
reply faster, and stay on top of every customer conversation
00:18:02.980
with a full conversation thread visible to everyone.
00:18:05.660
Quo works wherever you are, right from an app on your phone or computer.
00:18:10.200
It lets you keep your existing number and makes it easy to add teammates or new numbers as your business grows.
00:18:18.120
Quo's AI automatically logs calls, generates summaries, and highlights next steps so nothing gets lost.
00:18:24.320
It can even respond after hours, keeping your business responsive when you're offline.
00:18:29.520
Make this the year when no opportunity and no customer slips away.
00:18:32.920
Try Quo for free. Plus, get 20% off your first six months when you go to Quo.com slash Trig.
00:18:39.340
That's Q-U-O dot com slash T-R-I-G. Quo. No missed calls, no missed customers.
00:18:47.100
But why is it that the research into safety stopped? Why is that? Because surely,
00:18:55.140
I mean, I don't know anything about AI at all, but I listened to what you're saying and what
00:19:00.160
Lots of other people were saying, and I see this as an existential risk to humanity.
00:19:04.520
Why wouldn't you fund a very powerful AI safety board body, whatever you want to call it,
00:19:10.880
who will look into this, who are independent and assure that it doesn't affect our society
00:19:21.480
My argument is that it's impossible to do that.
00:19:24.760
You cannot indefinitely control something smarter than you.
00:19:27.980
So it's not a question of more money or more time or any other resource.
00:19:33.880
I think anyone who says, if you just give me a million dollars and more time, I'll solve it for you, they're lying to you.
00:19:47.540
No matter what changes we make to those systems, no matter who releases it, U.S., China, what company, what it's trained on, you want it to make zero mistakes.
00:19:57.320
Because if it makes one mistake, it could be the last one.
00:20:01.760
Just like perpetual motion would be impossible.
00:20:04.360
Your point is, a race of squirrels cannot indefinitely control a race of humans, effectively.
00:20:13.620
And so, no matter what controls the squirrels try and put in place, the very fact that humans
00:20:20.660
are a lot bigger and smarter than squirrels will inevitably lead to at the very least
00:20:28.180
the humans taking over right loss of control by squirrels is basically what you expect
00:20:35.140
and very quickly right yeah don't fancy being a squirrel in that situation personally
0.71
00:20:42.340
i mean humans had their chance we're screwing it up right now you seem very happy about this roman
00:20:53.940
it's kind of interesting to watch it happen like that like we know the right answers but we're
0.91
00:20:59.700
making the wrong decisions nobody makes an argument that they know how to control super
00:21:05.060
intelligence there is no company paper patent not even a good blog post yet billions of dollars are
00:21:14.100
spent to accelerate this process if prediction markets are saying we're four years away i'll
00:21:20.980
give you four years we have federal government saying we need to accelerate this project genesis
00:21:26.420
we're going to get more compute more scientists will make it happen sooner like in a week
00:21:31.060
i mean there are going to be positive elements to this aren't there when it comes to things like
00:21:39.560
medicine for example you know it may create the cure for cancer we can cure squirrel cancer
00:21:46.680
before they get wiped out yeah you know we can maybe it will it could be harnessed in order to
00:21:53.020
create a better life for the squirrels come on roman give me something here i think you can get
00:21:57.120
all those awesome benefits from narrow systems you can create a super intelligent cancer curing
00:22:03.040
ai one specific disease at a time you don't have to create general super intelligence so protein
00:22:09.120
folding example a very important problem in medicine tremendous impact was solved with narrow
00:22:16.640
system people who did it got nobel prizes more money for google everyone's happy let's do more
00:22:23.360
of that let's identify specific issues and have tools where a human decides to deploy that tool
00:22:31.600
to solve that problem not create a general replacement for all of human labor and humanity
00:22:36.960
as a whole so why aren't we doing more of that and why are we doing more general is it because
00:22:42.080
there's more money in general is it a power thing what's going on i suspect it's both so there is
00:22:47.680
definitely a lot more money if you make free labor cognitive and physical you're talking
00:22:52.720
10 trillion dollars what is it annually so that's a lot of money you can invest in it and still have
00:22:58.080
very good return no matter how expensive the current valuations are so that justifies the
00:23:03.440
current valuations people don't fully understand they only make 15 billion why are we investing
00:23:08.320
trillions into them or because they're saying in two years you'll get free labor and power is
00:23:14.000
another thing if they believe that someone's going to create it no matter what maybe if i'm
00:23:18.800
the guy who created god i'll get something out of it and do you think when you look at the big
00:23:24.320
figures in this world you know like the sam altmans look how much do you think they are motivated by
00:23:31.600
money and status and power and how much of it do you see is them wanting to be seen as you know
00:23:38.880
the people who created something transformative so in one of the blog posts i think he talks
00:23:44.480
about controlling the light cone of the universe. That's the level of power-seeking there.
00:23:50.960
Problem is, if I'm right and it kills everyone, you're not going to even be part of history as
00:23:55.360
a bad guy. There's not going to be history books. So they have more to lose than an average person.
00:24:02.240
And what do you think would be Sam Altman's steel man argument to what you were saying?
00:24:06.640
What would Sam Altman, if we were engaging in a debate, what would he say?
00:24:10.480
We'll figure it out. We have AI helping us do research now. Once we build it, we'll get there. We'll manage.
00:24:18.960
But that doesn't sound like they have any clear ideas.
00:24:22.160
That's official statements they are usually giving. We will have AI help us solve a problem,
00:24:28.960
or maybe it will turn out to be easier than we think it is. Those are the actual arguments we
00:24:36.160
we've heard so far. Because the concern is, when I hear about this, we had Jimmy Carr, the comedian
00:24:41.800
on, a few months ago, and he made the point that the barrier to entry with AI when it comes to
00:24:47.840
totalitarianism and mass surveillance is suddenly decreasing rapidly. If you think about East
00:24:52.800
Germany, you had to have the Stasi on every corner. You have to pay informants. All of a sudden,
00:24:58.380
you don't have to have any of that. It definitely has potential to lock in dictatorships,
00:25:05.000
But again, as long as it's a human dictator, we can look forward to them dying of natural
00:25:15.800
Once we lock in on a set of values, that's what you're going to have forever.
00:25:21.240
Yeah, I mean, all of these other concerns seem rather trivial in comparison to the thing
00:25:30.600
pause, though, and just set that to one side for the moment and talk about the replacement of
00:25:37.300
humans in the labor market, the impact in the interim period. Let's accept that, you know,
00:25:42.400
within 10 years, the superintelligence kills us all. Let's not accept it. Agreed. Agreed. I meant
00:25:49.740
for the sake of argument, of course. But in fact, for the sake of argument, let's say that you turn
00:25:55.740
out thank god to be wrong about that doesn't happen in the interim though we already see
00:26:01.420
people like to argue about this but to me it's just undeniable i know lots of business owners
00:26:06.480
who say constantin no no we're not laying people off we're just not hiring anyone and we probably
00:26:10.920
won't need to unless literally the people we currently employ die and then even at that point
00:26:16.240
we may not replace them we may not replace 10 people with 10 people we may replace 10 people
00:26:20.800
with five people, you know, what will be the impact of this in the next few years on the
00:26:27.260
labor market, on jobs, on the way economy is structured, et cetera?
00:26:32.340
So it's all about this paradigm shift from narrow tools to more general tools to a complete
00:26:40.080
We can define AGI as basically having a drop-in employee.
00:26:44.440
I can take someone, add them to the Slack, and then within weeks, we're starting to help
00:27:03.060
cognitive labor where you're a symbol manipulator,
00:27:14.960
So another three years, but we'll get there as well.
00:27:19.840
some jobs will be around because people prefer a human doing them
00:27:24.960
oldest profession is a great example sometimes you want to hear i don't know about that
00:27:29.520
i don't know try a robot but you want a human i don't know do you know i've been thinking
00:27:35.040
about this a lot okay everybody loves obviously yeah yeah but he's been thinking about sex robots
00:27:40.800
a lot tell us why yeah it's because my life is going really well anyway i'm single and
00:27:48.960
So anyway, because if you think about it like this, Roman,
00:27:52.740
you know, dating is hard, relationships are hard,
00:27:55.160
a lot of relationships fail, a lot of marriages fail.
00:28:02.920
put your heart on the line, all of that suffering,
00:28:06.460
when, let's say we get to this point where you can order a robot
00:28:10.760
and you can design her specifically to how you want every,
00:28:15.060
and let's not get into the details, but every single part of her.
00:28:18.520
you can also look at you can design her personality the spice level you like it a
0.58
00:28:24.480
little bit you know spicy or you like it however you want her to shout at you twice a month yeah
00:28:29.300
once a month exactly why would you put up with a human being who is erratic emotional is sometimes
00:28:37.860
unfair when you can literally have perfection as you demand it so there is a lot of weird human
00:28:44.580
fetishes pretty much anything you can think of there is a website for that somewhere on the
00:28:50.020
internet yeah and i guarantee you no matter how well sex robot market will be doing there will be
00:28:56.980
natural human females market fair but it might be a lot smaller i think is what francis is saying
00:29:03.940
it might shrink by 90 but when we talk about predicting unemployment i basically say that
00:29:10.020
almost everything will be 100 gone but few things will remain and this is one of the
00:29:16.500
last resorts we have as humans this is the career aspirations we'll have that i mean that's a
00:29:25.620
because one of the things that we talk about a lot on this show is the crisis of meaning
00:29:31.220
in this in our society where people struggle with what does it mean now to be a man what does it
00:29:36.180
mean to to be alive all of these things where once we had religion but this will introduce a
00:29:42.660
crisis and meaning the likes of which we've never experienced before i agree with that we call it
00:29:48.180
ikigai risks so ikigai is this japanese concept where you find happiness by doing something you
00:29:55.140
like something useful to society and something you're good at so you'll get paid for doing what
00:29:59.540
you like maybe you're a podcaster but if that is gone if there is no opportunities like that
00:30:05.380
then that takes a lot of meaning job so some jobs are just terrible nobody should be doing them
0.97
00:30:12.180
they're boring stupid we're happy to automate them other jobs give people satisfaction they
00:30:18.100
want to do more of them but they also would be automatable so this is exactly what we're facing
1.00
00:30:25.700
people make a counter argument well if i don't have to go to work i'll go fishing
00:30:29.140
there's eight billion people fishing in that lake right now you're not going to fish you know and
0.99
00:30:35.840
also we say we just to take your argument there are some jobs that are stupid or boring or whatever
0.98
00:30:41.740
else but when i was teaching there were i remember i had a child and he was being he was very low
0.94
00:30:49.800
ability he struggled at school and he found his lessons very very difficult and he would become
00:30:54.680
frustrated and he would lash out and i knew why he was doing that but nevertheless i had to
00:31:00.680
introduce some form of punishment to show him that his behavior wasn't acceptable and one day i kept
00:31:05.800
him in during a lunch break and i said to him marcus what you're going to do is you're going
00:31:09.880
to sharpen all these pencils so he went and sharpened all these pencils and when he came
00:31:15.400
back to me at the end of lunch break i thought he'd be upset frustrated angry and he had a look
00:31:20.120
of real pride on his face he went to me mr foster look at my pencils look at all the
00:31:24.440
pencils and they were all done beautifully and i realized at that point the reason he was proud
00:31:30.200
and it was my fault as well as everybody else's is for the one of the first times in his life
00:31:34.920
he had been given a task that he could succeed at and he could do and he could have pride in
00:31:39.800
i really worry roman that when you take that away from people we are all going to end up
00:31:44.840
like marcus lashing out angry and frustrated because we're not so different we still have the
00:31:50.600
child within us short term good news that is rent a human.com where you can get a job doing things
00:31:58.040
for bots so maybe we'll hire you to sharpen pencils there you go mate career sorted right
0.95
00:32:05.160
let's try counter argument um what if super intelligence creates endless abundance right
00:32:13.320
uh there are some problems with abundance the sorts of things that france is talking about but
00:32:17.880
you know park that to the side for one minute um and we humans are totally satiated by the
00:32:25.480
the productivity of i produces everything we could possibly want you know we've got wonderful lives
00:32:30.280
no one has to work blah blah blah blah and therefore humanity becomes sort of like you
00:32:37.640
know a nice pet for the for the ai to maintain to look after uh you know it's not quite ideal
00:32:45.480
but like you're a pet squirrel the eye looks after you it feeds you at the right time puts water in
00:32:51.080
your in your in your bowl um and it has no reason to not look after you because you're like its
00:32:57.960
beloved pet it could happen again we cannot predict what specifically would happen problem
00:33:04.920
is you are not in control sometimes owners decide to put you to sleep or neuter you or do other
00:33:12.520
things to pets you are not in charge so those decisions will no longer be with us we have 8
00:33:19.320
billion people who are not consenting to this experiment they cannot consent because they
00:33:23.400
don't know what's going to happen maybe you have your pet maybe you're abused pet we don't know
00:33:31.480
i'm struggling for counter arguments here i mean this this doesn't this doesn't sound good
00:33:35.800
um this is one of the better outcomes the safety angle where you're a pet you're protected you're
00:33:40.280
you're not in control. But this is one of the better outcomes. This is what people hope for
00:33:44.320
as a good outcome. This is what people hope for. Well, the other things are much worse. Existential
00:33:51.580
risk, suffering risk, all that is way worse. But if you're a pet, you literally have no agency.
00:33:59.040
Some people are very happy with that right now with the government.
00:34:02.760
That's fair. No, I think Romer's point is that it's not that people think this is the good
00:34:08.460
option they think it's the least worst option of the ones available normally my job on the show is
00:34:14.380
to interrogate the arguments that people put forward and try and find gaps but i've been
00:34:18.940
thinking about the same thing without having your knowledge of expertise in it and it does seem i
00:34:24.380
mean the very simple fact when you put survival instinct plus superior intelligence together
00:34:29.100
that seems to me inevitably to lead to the things you're talking about or at least to the very
00:34:33.180
serious risk of the things you're talking about okay and then i guess part of the reason it's
00:34:45.740
not getting solved is the collective action problem right that's why it's not being this
00:34:50.940
what is good for community is not what is good for individuals as an individual you want to have the
00:34:56.380
most progress on your model have the most advanced model and then if government comes in and says we
00:35:02.060
we need to stop research you forever are locked in as the dominant corporation in that space
00:35:07.320
i resisted creatine for years because i assumed it was for people who spend three hours a day in
00:35:14.380
the gym and refer to themselves in the third person turns out francis foster was wrong on
00:35:19.740
this one the research on creatine has moved way beyond the gym for years it was a preserve of
00:35:24.900
bodybuilders and sprinters but it turns out creatine is something your body makes naturally
00:35:30.260
and uses as fuel. Not just for your muscles, but for your brain, your energy levels, your mood,
00:35:35.800
your memory. The problem is that from your 40s onwards, your body produces less and less of it
00:35:41.620
and you feel it. The slower recovery, the afternoon fog, the sense that you're running on slightly
00:35:47.320
less than you used to. Talk to anyone who actually knows this stuff and they'll tell you the same
00:35:52.580
thing. Not all creatine is equal and most of it isn't doing what you think it is. The formula
00:35:58.740
matters specifically where it actually gets into your cells and activates once it's there that's
00:36:05.340
exactly what qualia creatine plus is built around two clinically studied forms of creatine combined
00:36:12.320
with electrolytes and sea salt designed to solve the whole problem not just half of it i've just
00:36:18.360
started taking it and once you understand how the formulation works you wonder why nobody built it
00:36:24.900
this way sooner. Go to qualialife.com slash Trig for 50% off and use the code Trig for an extra
00:36:33.140
15% on top of that. That's Q-U-A-L-I-A-L-I-E.com slash Trig, code Trig. Thanks to Qualia for
00:36:45.440
sponsoring the show. Thinking about the idea of China and the US in particular working together
00:37:06.940
is the same as it would be with nuclear weapons.
00:37:09.700
You have countries that say they don't have nuclear weapons
00:37:15.080
but actually because of the prisoner's dilemma situation
00:37:18.760
where it's to the benefit of each of them to screw the other,
00:37:22.400
to lie and then to develop the thing you almost some people would argue you can't take that risk
00:37:28.960
but then we're back where we started so there is a fundamental difference we talk about nuclear
00:37:34.400
weapons as weapons of mutually assured destruction but with ai with superintelligence it's literally
00:37:42.160
that whoever creates it uncontrolled supernatural kills everyone so it's not the same as with nuclear
00:37:48.960
weapons i have to decide to deploy them it's a tool i have an agent making this decision
00:37:53.440
the counterparty decides to retaliate we all die here just the fact that you created it is enough
00:38:00.160
there is no additional steps you have to take yeah and in you've been raising concerns about
00:38:09.120
this for a long time what has been the response from the leaders in the field of ai so leaders
00:38:18.000
of the labs are all on record as recognizing AI safety as a big problem. Before we became CEOs,
00:38:24.400
they wrote blog posts talking about it, estimating probabilities of doom as very high. And so they
00:38:30.960
are kind of on board. Like you can see example with Elon who was saying we are summoning demon,
00:38:36.960
funding AI safety research. So doing all the right things until somewhat recently.
00:38:45.840
He realized that he's failing to stop it and that others may be less capable.
00:38:51.680
People will be creating super intelligence. And at this point, it might as well be
00:39:01.040
Roman when I read about AI one of the areas that concerns me the most is when people who
00:39:16.460
program or started AI and look push back if I get this wrong I'm obviously not an expert
00:39:21.820
but it seems to me that when you program something you install your own biases within it even though
00:39:28.200
you may not be aware of having biases. Is there potentially an issue where people who program
00:39:35.700
a certain AI might make it more politically inclined one way or the other? Is that a real
00:39:42.880
concern? So you may program an AI which eventually bends more to an authoritarian angle or maybe more
00:39:50.820
hyper-conservative and therefore it sees these people as being wrong and evil for a particular
00:39:57.980
reason or is that a misfounded fear so a we are not programming those systems they are trained
00:40:03.820
on data the data has certain bias built in it's human generated data on the internet you know
00:40:10.460
what bias internet has so that's what we're training on to begin with now the after the fact
00:40:15.500
filtering is where you instill your corporate values and yeah they can be more vogue or more
00:40:21.980
conservative if you decided in china model would not talk about Tiananmen Square in US it would
00:40:27.100
not talk about you know what so everywhere they have their own limits elon i think is trying to
00:40:33.020
say let's build kind of truthful ai and avoid those biases but you still have the same training
00:40:40.300
data you don't have your own clean internet with clean data so you still get a lot of human
00:40:46.860
historical biases into that you can't remove all bias bias is what learning is when you learn
00:40:56.140
something you learn to bias data you're not randomly making decisions you have some information
00:41:02.300
a society we say oh this is not good information or it applies to groups not individuals or whatever
00:41:09.740
you decide but that's exactly what we train those systems to do and the concern for me is
00:41:17.500
is let's say you have a concern about the environment and you and the ai alights on that
00:41:27.500
and it says well you know the world is being damaged climate change pollution all of these
00:41:33.500
types of things these are bad things let's look at who causes the majority of the pollution in
00:41:39.180
the world human beings who cuts down the rainforest human beings therefore if you apply logic to this
00:41:46.380
problem. How do we solve this? Well, we get rid of human beings. Is that something that it could
00:41:52.420
be arrived at very easily? It's a good example. I have a different one where we create AI to
00:41:58.120
reduce suffering. Conscious life form suffers. So how would you reduce suffering in the universe?
00:42:05.100
Reduce life. If there is no living beings, there is no suffering.
00:42:09.380
there is a branch of philosophy negative utilitarians who value suffering so much as a
00:42:17.340
negative state anything should be done to remove it at any cost so not procreating for example is
00:42:23.800
one solution naturally dying out but ai can certainly decide that it's more important to
00:42:30.000
end suffering immediately and also as well you you see it more and more that people come to
00:42:36.720
ai as a de facto counselor or therapist presenting it with moral problems and this is becoming more
00:42:43.600
and more accepted and it seems bizarre to me that you would outsource very human problems
00:42:50.320
to something that is not human it's it seems to me that that is profoundly worrying isn't it
00:43:00.320
so we are kind of running experiment on ourselves we don't know what it does long term there's some
00:43:05.520
evidence that maybe it will take people who are borderline insane or depressed and amplify those
00:43:12.720
tendencies but we don't know we need to do science and we don't have time to do science properly
00:43:19.200
because by the time you start working with this model 20 new models have been released and this
00:43:23.680
one is no longer cutting edge one of the things that always bothered me about it and it was clear
00:43:31.280
in terms of the bias that you talked about because there was a moment when when you might say well
00:43:36.880
like most of social media was woke right and then now it you know some social media is not
00:43:42.000
woke quite the opposite right and the one thing that i think all of us know who live in the in
00:43:47.840
the real world is the internet is not real right but to ai that's all it has to go on right that's
00:43:57.440
all the data that it's taken in it's taking in this digital data which is not necessarily reflective
00:44:02.960
of human experience like if you were an alien coming down from space and someone said to you
00:44:07.520
the conversation happening on twitter or on threads is how humans think we humans would
00:44:14.960
laugh at that but ai doesn't know that does it right but it's not limited to internet data to
00:44:20.800
be fair it has all the books all the papers all the movies all the tv shows there is some
00:44:25.680
representation of real human interaction yes but we are sitting here in los angeles for example if
00:44:32.560
you watch hollywood and live outside of america your impression of america is not remotely accurate
00:44:40.240
because these films and series and movies are made by people who live in a very specific subculture
00:44:45.760
in hollywood um my point being that the human experience is a lot richer than what you can
00:44:52.720
gather from books and tv shows and the internet um is and i ai i think would is almost inevitably
00:45:01.840
going to miss that which would be another concern wouldn't it so this is where we can run experiments
00:45:06.800
and go okay you have a psychiatrist who's a model and a psychiatrist who's a human who does better
00:45:12.720
with clients who do clients like more apparently you don't have to have a physical body or be a
00:45:17.680
human to be very good at that job well being liked and being effective at different things right the
00:45:23.440
whole field is not effective how do you mean psychiatry psychiatry probably not yeah yeah
00:45:33.360
but there are types of therapy that are very effective
00:45:39.440
early studies show that those systems can do really well in many human domains so
00:45:45.840
So comments from nurses, things like that, they are competitive.
00:45:53.180
I was having something I couldn't work out what to do.
00:45:57.500
It was like, oh, yeah, you should do this.
0.91
00:45:59.220
And the thing about it I found interesting is it works best if you tell it not to bullshit you.
0.99
00:46:05.380
If you say to it, like, cut the bullshit, just tell me straight, it will do it.
0.98
00:46:15.840
So, one of the ways that AI is going to change the world is in the field of war.
00:46:27.040
And so talk to us a little bit how AI will impact warfare.
00:46:33.460
And what could be the future that we're heading towards?
00:46:37.460
So right now, it looks like it's more physical, mechanical, so you have drones blowing up
00:46:42.900
term it's more about cyber security hacking infrastructure so us has
00:46:48.960
everything basically controlled digitally right power plants internet
00:46:53.240
banking so if you had a super capable hacker that would be very impactful if
00:46:58.660
somebody wanted to attack us this way so just I think yesterday we learned that
00:47:03.420
Antropic has a more advanced model which is amazingly good at hacking and they
00:47:08.660
haven't released it yet and they are scared of how well it will do. And so we slowly trying to
00:47:14.820
release it to cyber defense community to figure out if they can do something with it. Because
00:47:21.620
correct me if I'm wrong, but if let's say you have a super hacker that is whatever 50 or 100
00:47:27.700
times more intelligent than even the best hacker, human hacker, it could render internet banking
00:47:35.440
entirely obsolete. I mean, what's the point of having internet banking if it's not secure and
00:47:39.780
it can get hacked at random? It could bring about the end of multiple businesses, the way we see
00:47:47.520
the internet, surely. Right. So you can obviously just hack things directly, find zero-day exploits.
00:47:53.760
What is also very concerning is social engineering attacks. If you can generate believable deepfakes,
00:48:00.720
video, audio, video from your boss, from your family telling you, I need a password for this
00:48:06.720
or click that. Everyone clicks. Even cybersecurity experts click and things like that. So you don't
00:48:12.580
have to hack the actual account. You just have to get access to the person.
00:48:17.040
And how far are we, do you think, are we away from that particular reality where you can get a call
00:48:23.320
and it could be sound like my dad, who's an older man from a particular part of the UK. And it's
00:48:29.240
exactly like his voice yeah so the technology exists and we've seen examples where the company
00:48:36.040
got video from the ceo saying transfer the funds i need them to close the deal and they transferred
00:48:41.320
the funds it already happened now it's not as common and easy for every person to do it on scale
00:48:47.720
but technology exists i can clone your voice i can definitely animate a video of you but it's not
00:48:55.400
quite trustworthy just yet so some people are not very good at telling deepfake videos from real
00:49:03.800
videos long term it will become impossible the quality will be exactly 50 50 because that's how
00:49:09.960
they are generated you have a system generating fakes and a system saying authentic or not and
00:49:16.680
they meet in the middle that's how we generate it using different models so long term there is no
00:49:23.480
way to know short term you can count fingers and sometimes it has an extra thumb or something but
00:49:29.400
most people don't pay attention to that but so could we not design a narrow form ai to actually
00:49:37.000
combat that and that is highly trained and very specific at those skills and we'll be able to go
00:49:42.920
that's real that's fake right so the moment you tell me how you know it's fake i'll use that
00:49:48.040
information to make it better quality fake and then we're done with this
00:49:52.540
process of back and forth you can't tell anymore so if you're telling me you just
00:49:56.980
count fingers and there is too many fingers my new model will make sure
00:50:01.000
there is five fingers so you lost that piece of evidence so it's a constant
00:50:06.940
what do you call it's like an evolution the war between the predator and the
00:50:10.060
prey that's exactly what it is an arms race this is an arms race an arms race
00:50:14.860
okay one more thing because the concern look there's many concerns of course it is of course
00:50:22.180
there is with that but the but the real concern is we're distorting reality pretty soon we're not
00:50:29.860
going to know what's real and what isn't how can i know if my dad calls me up he's like i've had a
00:50:36.560
fall i'm going to need some money we're going to need a put you know the we're going to need to go
00:50:40.900
to a private hospital i'm going to need however much to have a hip replacement i'm in the states
00:50:46.580
he's at home i'll go right transfer bang in my family we have private passwords so the kids
00:50:53.140
would know if i'm talking to them or deepfake it's terrifying because we're not going to know
00:51:02.340
what's real and what isn't and that has a dementing effect on ourselves because isn't that one of the
00:51:08.720
signs that you're going insane, that you can no longer trust what you see or what you think or
00:51:12.800
what you feel? It's all a simulation anyways. What do you mean? You haven't read my paper and
00:51:18.580
we all live in a simulation? We had Scott Adams on the show before he passed and he talked about
00:51:25.440
this, but not to us, I think. Is this why you're so serene about this, that you don't think this
00:51:30.060
is real? So when you take this technology to its logical conclusion, you will have software which
00:51:37.880
is intelligent agents you have virtual worlds they can reside in simulations of this planet like
00:51:45.080
google earth kind of deal put those two together you are now creating virtual worlds populated by
00:51:50.280
intelligent beings let's say all the kids are playing video games so there is four billion
00:51:57.160
virtual environments and only one real one statistically you are more likely to be an
00:52:02.680
agent in a virtual environment and i like this a lot because it kind of puts some doubt into the
0.97
00:52:11.240
mind of ai about am i being tested am i still in a simulation or is it time to kill humans
00:52:17.880
so i always try to promote that idea as well but humans are kind of badly designed like if
0.78
00:52:24.120
you were to design intelligence from scratch you wouldn't make it like need a every three
00:52:29.640
hours you know i mean maybe you would why fertilizer uh the easy ways to make fertilizer
00:52:38.600
not that easy but uh again you cannot criticize design if you don't know what the goals are
00:52:44.040
you cannot criticize simulation if you're not externally understanding the purpose of a simulation
00:52:50.840
look at our designs right i'm flying in an airplane some of them still have ash trays
00:52:57.880
why do they have ashtrays well airplanes evolved from previous version of it no no
00:53:03.240
it's not a poor design sometimes some decision was made for some reason you just don't know
00:53:07.640
all the history or the reason behind it this has not been the the most enjoyable episode we've ever
00:53:19.240
done but very important i'm really glad for you now i know what it feels like to be a woman all
00:53:25.880
I'll buy you dinner afterwards to make it up to us.
00:53:35.440
I prefer before, but afterwards will be fine as well.
00:53:40.940
This show exists because people want to hear ideas laid out properly.
00:53:50.740
And like any skill, it can be learned and improved.
00:53:53.340
which is why i recommend hillsdale college's new free online course classical logic and rhetoric
00:53:59.660
in this course a hillsdale college professor teaches you the tools to construct a sound
00:54:04.700
argument you'll learn how to think more clearly how to structure your reasoning so it holds up
00:54:09.420
under pressure and how to communicate your ideas in a way that people can understand and respond
00:54:15.660
to the course is part of a much wider library hillsdale offers more than 40 other free online
00:54:21.980
courses covering everything from the book of genesis and how the allies won the second world
00:54:26.780
war to the rise and fall of the roman republic and the american constitution all of it free
00:54:32.540
what draws me to this subject is the gap between how most people think they can argue and how sound
00:54:38.460
argumentation actually works rhetoric is not manipulation logic is not pedantry together
00:54:44.540
they are the tools that allow you to think and speak at your best this course makes that
00:54:49.500
accessible to anyone. To enroll, go to hillsdale.edu slash trigger. There's no cost and it's easy to
00:54:56.500
get started. That's hillsdale.edu slash trigger. So I guess the obvious question is, what do you
00:55:06.760
advocate that we now do? So it's easy. We don't do. Not doing is very easy. Don't build general
00:55:13.500
superintelligence. Don't train models on all the data, multimodal data, to solve every problem.
00:55:20.860
Concentrate on specific problems. Train only on relevant data. So you're talking about breast
00:55:27.660
cancer detection. Okay, great. Training that data. You'll have a superintelligent tool for doctors to
00:55:34.300
do early detection. You'll save lives. It's wonderful. I think you can get most benefits
00:55:39.980
from the economy with narrow tools and how do you achieve that not being done you know politically
00:55:48.140
and geopolitically what would it take for the u.s government or for the leaders of these companies
00:55:54.140
to to adopt that view personal self-interest if you tell the president of united states
00:55:59.020
the moment this technology comes around you lose all power that's a compelling argument i don't
00:56:04.380
think you would like that if this is the consensus of scientists in that field then maybe we should
00:56:10.700
not be building it and is it the consensus so if you look at the top three i believe computer
00:56:21.260
scientists by number of citations in the field they are in agreement this is really dangerous
00:56:26.620
it's not something we should be doing so we're talking about hinton who has nobel prize turing
00:56:32.460
Award, Banjo, Turing Award. I think we got maybe 100,000 people saying a letter saying
00:56:39.800
don't build superintelligence. Many top scientists. Are there outliers? Yes. Do they usually have
00:56:46.480
a company where they get billions of dollars to build AI? Also, yes.
00:56:50.560
Well, I was going to ask you about that because ultimately, you know the thing that maybe
00:56:57.440
I'm like this conversation is so wild to me that maybe my brain is like opened up to levels of
00:57:04.720
imagination that are not real but I'm just saying out loud what I'm thinking in the moment which is
00:57:11.280
two or three years from now the AI companies will be so powerful and I don't mean powerful in the
00:57:17.840
sense of money I mean powerful in the sense of powerful like the ability to kinetically
00:57:22.960
get what they want i'm not sure two or three years from now five years from now whenever
00:57:29.020
the president united states will be able to tell them stop doing this and unless they actually
00:57:36.020
agree to get them to stop so maybe nationalizing that technology will actually something we see
00:57:42.320
happen yeah but what i'm saying is like there comes a point where you actually physically will
00:57:47.740
not be able to nationalize them because they will be more powerful than the u.s government so again
00:57:56.620
You can come in, shut it down, change software, all that is possible.
00:57:59.920
The moment you're not dealing with superintelligence, it becomes a lot harder.
00:58:05.800
There's going to be a lot of people, regular people doing regular jobs with regular lives.
00:58:21.500
It's an unkind joke, but it's sort of like, I mean, that's what follows.
00:58:26.900
And they're going to think to themselves, look, if this is true, and there's no reason to believe it isn't,
00:58:33.980
this is all coming down the metaphorical pipeline.
00:58:37.220
What can I do to insulate myself as much as possible from this technology and my family's?
00:58:49.960
Some politicians are now starting to kind of wake up a little
00:58:53.180
and suggest we don't build maybe as much compute for those companies
00:59:00.500
But it's sort of like the whole concept of aging and dying.
00:59:14.180
Government didn't allocate funds towards that problem.
00:59:17.200
seems important in the same world you have like 90 of our budget going to fight aging
00:59:23.600
we're all dying so it's exactly the same scenario we just have a different
00:59:28.720
reason we're gonna die and maybe different timeline
00:59:33.520
depends on your age if you're like 95 it's the same yeah yeah well absolutely absolutely i mean
00:59:39.520
here's a question do you think it could solve the issue of aging and mortality could yeah in
00:59:46.160
a negative way. I think it's actually the narrow problem we should worry about. I think somewhere
00:59:53.600
in your DNA, there is a number of factors which allow you to rejuvenate yourself a certain number
01:00:01.520
of times. And if we can reset that number, you'd live a lot longer, much healthier life. Most
01:00:06.640
diseases are a byproduct of aging. And I think we can do it with a narrow superintelligence,
01:00:13.300
Because it seems to me that we are at a fork in the road now, right?
01:00:18.640
Where we can go down one way or we can go down another way.
01:00:22.980
And the worry is, is that we're heading down one way
01:00:38.680
don't understand that or I'm willing to see that?
01:00:46.080
They cannot say no to investors because they'll be replaced
01:00:50.880
The options are amazing, the stock options I get.
01:00:58.240
The hope is, again, that there is external pressure
01:01:00.700
for all the companies to stop at the same time,
01:01:03.460
and then they find they have an excuse to investors.
01:01:06.180
investors bought in at very high valuation they needed to go and have 100x so they needed to
01:01:14.520
continue growing hyper exponential towards super intelligence they cannot just say let's have normal
01:01:20.260
profits so the financial pressures are what drives them incentives are completely misaligned
01:01:29.280
we have no incentives which are pro-humanity all the incentives are to develop this do you think
01:01:35.680
part of the problem is as well, Roman, that the politicians don't understand the technology or the
01:01:42.500
long-term effects of this technology? So many don't, especially in US. Many are so old,
01:01:50.180
they don't use computers or internet or anything. Maybe they quit. I don't know. But we have some
01:01:55.860
politicians who are on record as saying, this is very bad, dangerous. We need to do something,
01:02:00.160
regulation. Problem is, you can't regulate this away. You can't just say it's illegal to kill
01:02:05.240
humanity. It doesn't work. You need to have specific bans on this particular deployment.
01:02:12.360
And I don't think they're willing to do that. And you need to orchestrate some kind of agreement
01:02:17.620
with China as well. I think that would be actually easier. I think that would not be the most
01:02:23.400
difficult part because, again, China doesn't have control mechanism. You think Communist Party wants
01:02:28.900
to lose control? They're very good at staying in control. And if they see this as potentially
01:02:33.960
threatening their long-term survival they'll be very happy not to do that that's an interesting
01:02:38.760
point you mentioned you have kids i do as well what do you i mean is there any point training
01:02:44.520
your kids to be able to do a job at this point well again it really depends on what type of job
01:02:51.000
i wouldn't train them to do something boring just to make money that's going to be automated anyway
01:02:55.960
so if there is something they find personally fulfilling to do so there is lots of things we
01:03:01.240
talked about one only human occupation but you can be uh i don't imagine you can you can do
01:03:08.280
all sorts of training you're a sensei you're a guide you're a tutor just human interaction
01:03:13.640
you take people on hikes you meditate you do sort of things where i don't want a robot doing it for
01:03:18.600
me yeah so it seems that we're going to prize human interaction above all else really i don't
01:03:28.200
know if that's true right now we're not value that much we sit at home and scroll so maybe we don't
01:03:33.400
need it as much as in terms of jobs i'm saying that certain jobs we will prefer to be done by
01:03:41.240
humans like which ones is not obvious podcasting i think if you are famous and you have people who
01:03:47.560
really like you you but i think i would be better at asking questions better at generating video
01:03:54.760
content so if you kind of grandfathered yourself in like you are joe rogan or something you'll be
01:04:00.040
okay but i think you could have just said trigonometry i mean who's that but i think
01:04:06.760
for a new person to start something like that successfully in a world with super intelligence
01:04:11.480
i see that editing and question because they watched every interview i ever did right they
01:04:16.840
know every question they write every paper how many of my papers have you read not many yeah not
01:04:22.760
right it's a good point i was thinking about this in terms of the kind of the political
01:04:30.200
element of it and i can really see roman 10 20 years down the line we get a kind of neo-luddite
01:04:38.520
movement which is anti-technology anti-ai and pushes back against that and it wouldn't surprise
01:04:47.560
me if we get also a terrorist element of this you know we will start for instance like i don't think
0.98
01:04:54.200
it will be long until you see people when waymos start taking people's jobs i don't think it'll be
01:04:59.720
very long until you walk past and by the way i don't agree with this i want to make this clear
0.98
01:05:03.240
you'll see a waymo with a smashed windscreen we just had the biggest ever protest to stop ai i
01:05:10.520
think in san francisco like 100 to 200 people showed up which is not a lot but it's a good
01:05:16.200
starting point. If you're interested in this social unrest civil war, Hugo de Garis has a beautiful
01:05:23.640
book, Artilect War. He wrote it like 20 years ago, completely predicting all these elements.
01:05:29.000
This is the most important issue of our time. There will be people who want to create godlike
01:05:33.400
machines and go to cosmos and people who are Terrans. They want everything to be local and
0.99
01:05:39.320
not to build those machines. And that's the decisive issue of our time.
01:05:44.360
because we talk about mass surveillance states the government would say well look you know
01:05:51.960
more and more particularly young men are unemployed they can't get a job because
01:05:58.280
the jobs that they used to be able to get like driving jobs manufacturing they've all gone
01:06:02.680
they've all gone so we've got this large group of unemployed young men and if they don't have
01:06:07.480
a job what tends to happen is they get angry they get more violent and the government will then come
01:06:13.160
in and go well look we've had all of these civil unrest and uprisings and riots we can't have this
01:06:20.360
therefore it's very important that we bring in mass surveillance to keep you safe i mean that's
01:06:25.000
a real possibility isn't it it is possible without concern about super intelligence we just have
01:06:31.400
governments deploying latest technology to spy on us we're seeing it with snowden we're seeing it
01:06:35.880
with others revealing what's what's really happening right yeah and it's also the concern
01:06:41.080
as well that we're going to live in a world which is far more unstable because we have these large
01:06:46.200
groups of men who don't have access to a job so the economic part of not having a job is easy to
01:06:53.480
solve you can tax big ai you can tax robots and distribute that that's not the difficult part
01:07:00.440
meaning is difficult yeah control is difficult yes roman well um thank you i guess for coming
01:07:10.120
on the show now we're very grateful for your time i'm just i think um unfortunately you've confirmed
01:07:17.400
a lot of the things that and i was wondering about this you know you i think from the former
01:07:21.880
soviet union i am francis you know he has some family ancestry from countries that have had
01:07:27.320
difficult uh existences i i always worried that it was my kind of temperamental russian
01:07:33.800
background that makes me worry about this stuff. But as always, when I don't see a logical
01:07:41.880
counter-argument, that's when I go, well, until I hear one, I will think this is likely. And
01:07:48.960
I don't see the counter-argument to the very basic point you're making, which is if you're
01:07:55.060
a squirrel, you cannot keep humans under control. And anything that has a survival instinct that
01:08:02.760
don't control that's more intelligent then you will eventually take over best case scenario
01:08:08.120
best case scenario so the reason we are kind of uncomfortable is that like this is um this has
01:08:15.000
kind of become real for us in this conversation so thank you for coming on i hope more people
01:08:20.040
hear your message and um humanity begins to to take this seriously i hope so so usually in science
01:08:28.600
when you publish a paper or a book and you are wrong there is no shortage of people jumping in
01:08:33.800
and publishing rebuttals corrections solutions we have many papers many books all arguing the
01:08:40.760
same thing there is no rebuttals there is no patents there is no peer-reviewed papers in nature
01:08:46.280
saying this is how we control advanced ai it scales to any level don't worry about it
01:08:50.840
so it's not just that we had this conversation and so far nobody jumped in they had a decade
01:08:58.600
Thank you so much for coming on. Before our audience asks you their questions,
01:09:03.000
the last question we ask all of our guests is what's the one thing we're not talking about
01:09:06.860
that we should be? Before Roman answers the final question at the end of the interview,
01:09:11.520
make sure to head over to our sub stack. The link is in the description where you'll be able to see
01:09:16.080
this. Is there an argument that humans are now in service of a new form of organism without
01:09:21.880
realizing it? Do you think there is a risk that AI leads the human race becoming complacent,
01:09:27.240
not bothering to study research and advance ourselves what's the one thing we're not talking
01:09:32.780
about that we should be suffering risks suffering risks tell us more so things could be so bad you
0.51
01:09:39.840
wish you were dead why digital hell you can create environment where you are tortured but you are
01:09:49.900
immortal or maybe you are uploaded to a virtual environment and what for you're asking too many
01:09:56.920
questions super intelligence can decide to do all sorts of things maybe it's dealing with some
01:10:03.960
malevolent payload maybe it's running experiments you can ask why this world this simulation has
01:10:10.840
suffering in it right that's what every religion deals with why did all good god create a world
01:10:17.560
with pain and suffering but there are some answers to those questions and it's not ruled out by
01:10:26.760
what we coded into those systems nice to end on a positive all right at least we didn't talk
01:10:34.760
about this again at least we didn't talk about it yeah that's the question head on over to
01:10:40.440
triggerpod.co.uk where Roman is going to answer your questions in the best case scenario where
01:10:47.560
ai doesn't erase us is it plausible that humanity avoids becoming merged with technology directly