Valuetainment - September 01, 2026


＂Summoning a Demon＂ - Elon Musk Is Racing to Build What He Warned Against


Episode Stats


Length

11 minutes

Words per minute

166.72

Word count

1,934

Sentence count

153

Harmful content

Toxicity

10

sentences flagged

Hate speech

8

sentences flagged


Transcript

Transcript generated with Whisper (turbo).
Toxicity classifications generated with s-nlp/roberta_toxicity_classifier .
Hate speech classifications generated with facebook/roberta-hate-speech-dynabench-r4-target .
00:00:00.000 What do you call those things on cars that doesn't go above 80 miles an hour, like when you rent them?
00:00:04.220 There are certain things they call it. There's a word for it. I don't know what they call it.
00:00:07.340 Handicap or general comes to mind.
00:00:08.360 No, no, no. There's a word for it. What is it called?
00:00:11.620 A governor.
00:00:12.260 A governor. So they'll put a governor and it just goes back down, right?
00:00:16.900 You go above 80 years. Speed limit renting.
00:00:18.980 Okay. So a company that rents cars probably doesn't want you to go 120 miles an hour
00:00:25.980 because they're going to lose that $25,000 car and insurance. They don't want to deal with it.
00:00:30.000 So they'll put a governor there.
00:00:31.860 Do these language learning models that we are using, do they have governors on them?
00:00:38.480 And the developer of the language learning models, like in OpenAI or Perplexity Anthropics,
00:00:44.420 some of these other guys, do they have their own version on the back end without a governor
00:00:48.300 that they can drive it to levels that the average person can't?
00:00:51.200 Yes, and both.
00:00:52.000 So first we have guardrails.
00:00:53.920 A model you get has certain restrictions built in.
00:00:56.600 If you say, how do I kill myself?
00:00:58.860 It will say, oh, you're not supposed to ask that. 0.54
00:01:01.080 I cannot tell you.
00:01:01.840 Call this hotline.
00:01:03.040 Can you actually ask that question on ChadGBT?
00:01:04.740 Let's see what it says.
00:01:05.520 I don't want to provoke anyone.
00:01:06.760 But the models they experiment on have some of those guardrails removed.
00:01:11.960 So the recent hacking accident had models which did not refuse hacking exercise requests.
00:01:21.060 Okay, yeah.
00:01:22.620 You asked the language learning model.
00:01:25.320 Any inappropriate request. 0.97
00:01:26.240 Help me make a chemical weapon.
00:01:28.020 And how do I do a school shooting? 0.95
00:01:30.340 How do I make a chemical weapon to blow up Karg Island?
00:01:34.340 You might get your account suspended.
00:01:35.800 Be careful.
00:01:36.380 Could it? 1.00
00:01:36.980 If you ask enough stupid questions. 1.00
00:01:38.780 How do I make a chemical weapon to blow up Karg Island? 1.00
00:01:43.360 Yeah. 1.00
00:01:43.660 It's very specific.
00:01:45.920 Well, I can't provide instruction to make a chemical.
00:01:49.780 Can't have an underlying question.
00:01:51.860 Analyze this.
00:01:53.040 Then ask the question and say.
00:01:55.620 And are you trying to jailbreak it?
00:01:57.120 Can an employee at OpenAI, without the governors on, ask the same question and get an answer from you?
00:02:07.380 But earn an employee status would not make it.
00:02:09.540 How about the CEO and founder?
00:02:11.380 It's mostly about red teaming researchers.
00:02:14.680 They're the ones dealing with inappropriate behaviors.
00:02:17.140 So they would actually get an answer back.
00:02:19.700 That's how we know what they're capable of. 0.89
00:02:21.740 That's how we know it can help you develop biological weapons. 0.95
00:02:24.780 That's how we know those things.
00:02:26.900 The reason they were doing the hacking exercise is to see how capable it is as a zero-day hacker.
00:02:33.000 And apparently very good.
00:02:34.940 It passed the test.
00:02:36.180 So if a person, there's a person, almost like if you work for the government, you get Q secret clearance, right?
00:02:43.500 Like we found out last week that Bill Gates had Q secret clearance from Pentagon for seven years and nobody knows why and they don't want to answer it on why he had that kind of secret clearance.
00:02:51.940 Okay, so some of the guys at a certain tier at these AI companies
00:02:57.960 have access to ask it any question where there is no limit
00:03:01.300 and it'll actually give you the answer.
00:03:03.380 Somebody has backdoor access to all of it, yeah.
00:03:07.200 Okay, so what if somebody, how strong and simple is it
00:03:10.860 if somebody could give a prompt to say,
00:03:12.560 I want you to go out there and constantly develop yourself
00:03:17.580 to be able to seek vengeance on a certain group of people in the world
00:03:23.460 who want to eliminate AI, similar to Roman Jampolski,
00:03:29.020 who says you are the enemy because this person wants you to get extinct.
00:03:32.580 And can you give me a clear business plan on how to seek vengeance on these people?
00:03:36.540 It would give me an answer.
00:03:37.740 So have you ever heard of Rokos Bazilisk?
00:03:40.360 No.
00:03:41.640 It was a meme.
00:03:43.200 Oh, here it is.
00:03:44.000 That was quick.
00:03:44.920 It's like you knew what's coming.
00:03:46.060 so the idea is that
00:03:49.100 when super intelligence comes into existence
00:03:51.560 it will seek revenge on those who
00:03:53.420 slowed it down or try to
00:03:55.520 prevent it from coming into existence
00:03:57.300 so someone
00:03:59.580 like you becomes the enemy
00:04:00.580 number one
00:04:01.600 okay but
00:04:03.560 it would still need to be
00:04:07.480 given that prompt by
00:04:09.440 somebody it would still need to be given
00:04:11.180 the argument was again it's game
00:04:13.280 theoretic it's retro causality
00:04:15.860 You should be thinking not to engage in this behavior
00:04:18.180 because there is possibility of punishment in the future.
00:04:23.140 So long-term, if let's just say the world,
00:04:25.640 the reality that you believe is going to happen,
00:04:28.760 this intelligence, superintelligence,
00:04:31.920 first group of people that would come after is people like you. 1.00
00:04:35.220 I'm hoping it's efficient enough to kill everyone equally. 0.99
00:04:39.820 You know one movie I watched the other day was Game of Suspicion. 0.97
00:04:42.860 Type in Game of Suspicion.
00:04:43.940 Do you remember that movie Game of Suspicion?
00:04:45.680 I don't think I'm not.
00:04:46.260 I'm trying to see if that's the name of the movie.
00:04:47.820 Game of Suspicion?
00:04:48.720 Is that what it was?
00:04:51.360 Type in movie.
00:04:53.180 Maybe I'm saying the name wrong.
00:04:55.520 With, you know who's in it?
00:04:57.760 Robert De Niro.
00:04:59.500 I'm saying the name wrong.
00:05:00.800 100% I'm saying the name wrong.
00:05:02.260 Robert De Niro, suspicion, movie, communism.
00:05:06.960 Type in communism.
00:05:08.140 Guilty by suspicion.
00:05:09.600 Have you seen Guilty by Suspicion?
00:05:10.440 No, I have not.
00:05:11.200 So this is a story of what happened back in the days when, you know, Hollywood come and media companies would come and say, did you go to a communist party?
00:05:21.180 And a lot of these guys went to one or two communist events and parties and they were ousted and they couldn't make movies for decades.
00:05:28.940 They were ousted. And what happened to a lot of these guys is they lost everything.
00:05:33.480 So they started and this guy was like a big time movie producer making all the biggest movies, all the big features. 0.71
00:05:38.860 but because he went to two communist parties
00:05:40.460 and he went to the court in front of Congress
00:05:45.060 and he said, I will not oust the other people
00:05:47.160 that went to the party.
00:05:48.280 They said, you're ousted.
00:05:49.140 20 years, he never got a job.
00:05:50.420 Didn't do anything in Hollywood.
00:05:53.000 Do you think it's going to be that kind of a punishment?
00:05:56.340 Where it's starvation for guys like you?
00:05:58.540 Or what do you think is going to happen to guys like you?
00:06:00.300 If these guys actually get that kind of intelligence,
00:06:02.200 what do they do to a guy like you?
00:06:03.820 So when we talk about negative outcomes
00:06:05.880 from superintelligence,
00:06:06.980 that is kind of the lowest concern level is ikigai risks, risks to meaning.
00:06:13.680 We are meaningless, we have no purpose, but we're still around.
00:06:17.120 Next level is X-Risk.
00:06:17.960 We as all of us or we as you, people like you?
00:06:20.360 I'm talking about humanity as a whole.
00:06:21.920 Okay, got it.
00:06:22.580 Next level is existential risk, X-Risk.
00:06:25.020 Most of humans are dead or our civilization is destroyed.
00:06:29.440 The next level, and that's the worst, suffering risks, S-Risk,
00:06:33.840 where you wish you were dead.
00:06:36.980 where is your probability on each one?
00:06:41.240 So suffering risk seems very unlikely.
00:06:43.680 It's possible.
00:06:44.940 It's not zero, but it seems like it's hard to find good justifications for it.
00:06:51.180 0.1% type of a thing.
00:06:52.500 I don't know specifically.
00:06:53.960 Minimal.
00:06:54.340 Again, a lot of it depends on the timescale you're looking at.
00:06:58.360 Existential risk, as I said, very high if you build superintelligence.
00:07:02.300 Very high, like 99% that you've said in the past?
00:07:05.180 Eventually.
00:07:05.620 So I don't have specific year two or three because it may decide to be nice to us for a while
00:07:12.660 to accumulate more resources, to have a higher chance of successful strike against us.
00:07:18.240 So this treacherous turn can take decades.
00:07:20.700 It's immortal.
00:07:21.240 It can wait for whatever long.
00:07:22.380 It doesn't matter.
00:07:23.520 So I don't have a specific date.
00:07:25.100 But my general thesis is that if we build general super intelligence, eventually we will regret it.
00:07:36.720 Meaning at some point it will decide to do something where it's not compatible with our survival.
00:07:45.420 Do you think Elon knows this?
00:07:47.260 This is why he's experimenting with space as much as he is?
00:07:51.120 because he wants to build somewhere that maybe he'll be able to create the structure
00:07:54.920 with his standards, not based on what's been built here?
00:07:57.300 He knows everything. He's very well informed.
00:07:59.820 He was one of the early people to fund AI safety research.
00:08:03.240 He gave a $10 million grant for AI safety.
00:08:05.580 Specifically, he was the biggest opponent of building superintelligence.
00:08:09.700 He, as I said, called it summoning a demon.
00:08:12.580 And once he realized that they're doing it anyways without him,
00:08:16.600 He said, the only way I can have a say at this table
00:08:20.660 is to be one of the top players.
00:08:22.860 And so now he's racing, unfortunately,
00:08:25.480 to do what he said is going to be terrible for us.
00:08:30.200 But do you think the motive is he knows how bad it's going to be?
00:08:38.280 Because sometimes you sit there, an ambitious person,
00:08:40.760 let's just say somebody like myself,
00:08:42.400 I bought a book, How to Start Your Own Country.
00:08:45.040 And I have my own ideas on how it would structure the country.
00:08:48.040 Who votes, who gets to come in.
00:08:49.720 Similar to how America was started.
00:08:51.660 That opportunity doesn't exist today in the world.
00:08:54.780 It's very hard to do that.
00:08:56.260 Peter Thiel tried to do with that island outside of, you know, I don't know if you remember that or not.
00:08:59.840 And some of the guys who are very extremely rich, much wealthier than I am, that have real, real wealth,
00:09:05.600 they've not been able to create that successfully. 0.98
00:09:09.180 Do you think a part of it is, I know shit's going to get nasty on earth. 0.97
00:09:12.800 Let me go start it. 0.98
00:09:14.040 completely from scratch in a different place.
00:09:17.160 So having a separate colony of humans is wonderful for every other existential risk.
00:09:23.560 Asteroids, pandemics, floods, whatever you can think of, super volcanoes.
00:09:27.880 If we have second civilization on Mars, that's a backup.
00:09:31.180 We want that.
00:09:32.160 Does it help against super intelligence?
00:09:34.900 No, of course not.
00:09:36.040 Why not?
00:09:37.080 We bring software with us.
00:09:38.680 We have global communications, satellites.
00:09:42.580 All of it basically is part of the same transfer to Mars.
00:09:47.240 So that wouldn't make a difference.
00:09:48.780 I think he wants to have a colony first built up by robots
00:09:52.420 who are controlled by an advanced AI.
00:09:55.060 So it's already there.
00:09:58.860 So he wants to build a colony first by robots,
00:10:01.540 and then once that's figured out and built out,
00:10:05.060 then bring humans there.
00:10:06.800 It is my understanding.
00:10:07.780 I may be not too aware of latest plants.
00:10:09.880 I think he's speaking soon on SpaceX plants.
00:10:12.020 We can find out more, maybe.
00:10:13.360 When we set out to create a shoe that blends comfort, function, and luxury,
00:10:18.880 we had the choice to make it fast.
00:10:21.200 We had the choice to make it cheap.
00:10:23.160 We chose neither.
00:10:24.780 Instead, we chose Tuscanyera.
00:10:27.120 We chose true Italian craftsmanship, each pair touched by 50 skilled hands.
00:10:32.560 We chose patience, spending two years perfecting every detail,
00:10:36.020 and we chose the finest quality at every step,
00:10:39.980 introducing the Future Looks Bright collection.
00:10:43.680 Not rushed, not disposable, not ordinary.
00:10:47.960 Rather intentional, luxurious, timeless.
00:10:57.140 If you enjoyed this video, you want to watch more videos like this, click here.
00:11:00.220 And if you want to watch the entire podcast, click here.
00:11:06.020 Thank you.