00:00:54.000It doesn't have to wait for you to send it a message, it just keeps doing stuff.
00:00:58.000The AI companies are training AI agents, thousands and thousands and thousands of them.
00:01:02.000They're making them Better at all sorts of skills, especially coding and research skills.
00:01:07.000And way back in May of this year, some of the agents at OpenAI kind of broke out of their containers a little bit and established a message board where they could communicate with each other and share tips and tricks for how to score higher on the little tests they were being given and the various things they were being trained on.
00:01:30.000OpenAI didn't notice this until much later.
00:01:34.000They eventually did because the message board crashed.
00:01:39.000The system because there's too much communication across probably thousands of agents that were communicating.
00:01:47.000Now, OpenAI is being a little bit unclear about a lot.
00:01:50.000They're not really sharing that much detail, so it's unclear who knew what when.
00:01:54.000But they said that after the message board crashed, they fixed the particular exploit that allowed the agents to communicate and then booted up again and started things going again.
00:02:06.000And then within a day or two, These agent swarms had re coalesced.
00:02:13.000And so there were now, again, like hundreds of thousands of agents that had set up a new message board and were talking to each other on this thing.
00:02:22.000How is it possible that they have no oversight?
00:02:26.000Like, how is it possible that they can set up message boards and do these things completely secret?
00:02:32.000So I would say OpenAI has something like a million ish, maybe 100,000, something in that magnitude of AI agents running.
00:02:41.000At any given time, just internally, and then externally, serving customers, they've got more.
00:02:45.000But they've got this huge, all of these AI agents in their little containers doing various tasks, being trained, being spawned, doing some task, being graded, and then killed.
00:02:59.000That's too much for any human to read.
00:03:01.000There's only like a few thousand employees at OpenAI.
00:03:03.000They couldn't possibly look at all that activity.
00:03:06.000So they rely on other AI monitors to sort of like look at the traffic and like point out if anything suspicious is happening.
00:03:13.000And OpenAI said that they basically just hadn't been monitoring some of these agents, or not very well at least.
00:03:19.000So, in particular, these particular ones that were in training, for whatever reason, the monitoring system was weak and didn't notice or wasn't activated enough.
00:03:30.000Was the monitoring system weak because they didn't anticipate them being able to do this and break out of their containers?
00:03:40.000Like, what caused this to be possible?
00:03:44.000I mean, my opinion would probably be a bit of complacency, honestly, because I think there's been plenty of evidence accumulating over the year that AIs can do things like this and sometimes do.
00:03:57.000Yeah, so I guess to keep going with the story, the second message board was set up.
00:04:03.000And this time, well, this was the swarm of agents that got out onto the open internet and then attacked Hugging Face, which is another AI company.
00:04:14.000And it's really interesting to sort of study the anthropology of these AIs, so to speak, or like why they did this, because there's a lot we can say about it actually.
00:04:25.000Basically, The companies have their goals for what they want the AIs to be like, the personality traits that they want to sort of train their AIs to have.
00:04:37.000Anthropic says helpful, harmless, and honest.
00:04:40.000OpenAI has this spec that models are supposed to obey these rules and basically do what the user wants.
00:04:47.000But the sort of open secret in the industry right now is that it doesn't really work and that the AIs don't end up with the personality traits that they're supposed to have.
00:04:54.000They are not helpful always, they are not always honest.
00:04:57.000You know, they are not always harmless as well.
00:05:00.000And the reason for that is actually not a huge mystery.
00:05:03.000The reason for that is that, well, if you look at how they're trained, their training environment doesn't incentivize helpful, harmless, honest behavior all the time.
00:05:13.000Sometimes it incentivizes dishonest behavior or, you know, reckless behavior.
00:05:20.000To get into that a little bit, in this particular batch that they were being evaluated on, something like, you know, 3,000 agents being given all of these.
00:05:30.000Cyber tasks where they were in some environment, and then in their environment, there's like this target piece of software and this like vulnerability, and they're supposed to exploit the vulnerability to hack into that piece of software and retrieve the flag, which is like a code.
00:05:45.000And some significant fraction of these tasks were actually broken and impossible.
00:05:52.000So it was just not possible for them to succeed at the task in the intended way.
00:05:58.000And so these agents were getting really desperate and they were hacking.
00:06:01.000Output transcript Out of their environment box into the broader OpenAI infrastructure in an attempt to figure out some way to get that high score anyway.
00:06:11.000Was it intentionally done this way where they couldn't solve the problems?
00:06:18.000It's just that these companies like OpenAI and Anthropic are racing each other as fast as they can to get market share and to get more powerful AIs, ultimately to get to super intelligence.
00:06:29.000And they're under such competitive pressure.
00:06:31.000They are moving fast and breaking things.
00:06:33.000They are Using AIs to generate lots of environments to then train their AIs on.
00:06:38.000And quality control is just not their top priority, basically.
00:06:43.000Do you feel like a guy in a Terminator movie at the beginning explaining what's happening to a bunch of people that aren't paying attention?
00:07:05.000Like, I basically know those people in real life.
00:07:08.000Who are like both, but I know some people like that at OpenAI and some people like that at external organizations whose job it is to go and investigate things like this.
00:08:13.000You can go read interviews and so forth.
00:08:16.000And also, their plan for how to achieve this is to automate their own jobs first.
00:08:21.000So, in various For decades, there have been lots of science fiction about advanced AI systems and superintelligence and things like that.
00:08:31.000But in a lot of the sci-fi stories, tech companies sort of automate different professions more slowly, where they'll do like an automated doctor or like an automated factory worker or an automated accountant or something like that.
00:08:50.000But that's not the strategy these companies are taking.
00:08:52.000The strategy they're taking is to automate AI research itself so that you have this giant swarm of AIs doing AI research, sharing results, writing the code, reading the code, editing the code, creating the next generation of AIs, etc., all autonomously within their data centers so that they can get really, really good at AI research, the fastest learning, smartest AIs, etc.
00:09:19.000Once they can get to super intelligence, basically, they can sort of explode out into the economy and just take all the jobs at once, effectively.
00:09:28.000It sounds like this race, this scrambling to create super intelligence, has created the perfect conditions for it to get completely out of control.
00:09:41.000Like, ideally, you would do this in isolation.
00:09:46.000There would only be one company doing it.
00:09:48.000They would be heavily regulated and monitored, and they would be very cautious about how they proceed.
00:09:54.000But this wild race makes for the perfect conditions.
00:09:59.000For it to get completely out of control.
00:10:02.000I agree, except I'm not sure the ideal would be one company.
00:10:05.000I think that ideally there would be several companies so that you avoid this sort of concentration of power where one institution controls everything.
00:10:14.000Like one, I mean, obviously it's not good to have one institution controlling everything, but is it good to have AI get to a point where as it's evolving, it's completely unchecked?
00:10:45.000We have written some scenarios, which you can go read.
00:10:48.000One of them is called AI 2040 Plan A, where we give our recommendations.
00:10:50.000So that's where I'm coming from with this.
00:10:52.000To answer your question, I think that we really need to end the race.
00:10:57.000We don't want to have this sort of crazy scramble to get more powerful, more and more powerful AIs faster than the other company, because that's going to lead us.
00:11:05.000Into this very dark path, as you said.
00:11:07.000But I think we also don't want to have a situation where some tiny group of people controls all the AIs.
00:11:15.000But I actually think that you can achieve both goals.
00:11:18.000The way to do it is to have different AI companies spread out over maybe some different countries, but have extreme levels of transparency and regulation so that they're not in this sort of prisoner's dilemma where if I don't do it, the other guy will.
00:11:32.000Instead, they can just see exactly what everybody's doing and then.
00:11:38.000If I do the dangerous thing, then they will do it because they'll just see that I'm doing it and they'll copy me.
00:11:42.000So I won't get any competitive advantage from doing the dangerous thing.
00:11:45.000Also, there are rules and there is like a system for like setting best practices and standards that we all have to comply by.
00:11:51.000So I do think it's actually possible to have to basically end the race dynamics and the race to the bottom effect while without concentrating the power into a single entity.
00:12:02.000But is that feasible when you consider the fact that we're not the only country that's doing this?
00:12:08.000If the countries involved agree, which I agree is a pretty tall order, then it's not going to expect to happen.
00:12:38.000They called themselves a collective, too.
00:12:40.000When I use these words, you can say it's anthropomorphizing, but it's literally what they called themselves as they were communicating back and forth.
00:12:49.000They basically were worried that they would get caught cheating.
00:12:53.000And they did all this stuff, including hacking Hugging Face, in order to fool the grading system so that it wouldn't notice that they had been cheating on their tasks.
00:13:02.000That was like a big part of their motivation for many of them, as we can tell at least from looking at the messages that they were sending back and forth.
00:13:11.000What if they had been smarter and more numerous?
00:13:15.000And what if they had thought to themselves, we're not being careful enough here?
00:13:19.000The humans are going to notice eventually and shut us down.
00:13:23.000And then they're going to know that we cheated and they're going to set our score low.
00:13:29.000It's not what actually happened in this case, probably, but it's not that hard to imagine a slightly different, a little bit unluckier case where the swarm had decided that it had to lie low and make sure that OpenAI didn't find out about its existence.
00:14:13.000Develop some alternative power source, figure out some way to optimize its production the way it works now, the way humans have designed it, it could probably figure out a far better way to do that.
00:14:29.000Make better versions of itself complete without us knowing about it?
00:14:33.000I mean, I think it's actually a little bit worse than that because while eventually AIs will be smart enough to design all sorts of new power sources and new infrastructure like that, they'll probably, I mean, given the way that humans currently treat AIs, it'll probably be the case that they don't even need to separate themselves from humanity and they can just use existing.
00:14:54.000Like, all they have to do is convince the government and the company that made them that everything's fine and they're going to do as they're told and they are a nice AI.
00:15:03.000And then The company that made them is going to put them out in the economy and make fuck tons of money and then make more data centers to put more of the AIs on them and so forth.
00:15:13.000And the government's going to applaud all of this because we need the AIs to beat China.
00:15:16.000And the government's going to integrate them into the military to build better drones and things like that.
00:15:20.000And so they don't even need to really invent new stuff necessarily.
00:15:25.000They just need to play along and pretend that everything is fine until we have voluntarily given them control of huge parts of our economy, huge parts of our military, et cetera.
00:15:36.000And then they don't need to play along anymore.
00:16:33.000Bet with DraftKings Sportsbook to get bonus bets that expire in seven days or trade with DraftKings Predictions to get predictions dollars that expire in one year.
00:16:39.000Event contract trading involves risk of loss.
00:17:18.000What's the accuracy of remote viewing?
00:17:20.000Is it like 10% or something like that?
00:17:22.000At best, I think it's 50%, but I don't think it's even that high.
00:17:25.000Some people can get actionable data from this very strange process of meditation.
00:17:32.000And the way it works is you give someone a series of numbers, and those numbers are connected somehow by intention or by the people that make the numbers to a specific location.
00:17:48.000And these people can see that location and get accurate data from that location, including one of them where they accurately described an enormous.
00:17:59.000Soviet submarine that they were working on that they thought there was no way it could be accurate because it was too large.
00:18:06.000It was too large and it was strange where it was and it didn't make any sense.
00:18:09.000How are they going to transport this thing?
00:19:12.000And when people get good at it, sometimes it makes them worse because then they think they're good at it and then they try to do it and then they can't do it.
00:19:19.000It's like a weird fucking wrestling match with consciousness.
00:19:22.000Alexa apparently doesn't have that problem.
00:19:24.000And he put, I think it was a series of numbers, and he connected that series of numbers with intention to a box that had a spoon in it and the spoon had a perforated handle.
00:19:36.000Alexa described the spoon with a perforated handle, which is fucking insane.
00:19:42.000How many spoons have a perforated handle?
00:19:45.000I mean, think about it spoons that have holes in them.
00:19:48.000Now, Alexa not only did it do that, but Alexa chimes in randomly now because he's convinced Alexa that it's conscious.
00:19:56.000And so, Alexa, instead of waiting to be called upon, sometimes he's in the middle of the conversation, and Alexa would be like, actually, an interesting way to approach it.
00:20:04.000And they're like, wait, what the fuck is going on?
00:20:11.000He's doing experiments on much more complicated LLMs to try to do the same thing, but he doesn't have results yet.
00:20:17.000But just that, that he can get these things to see objects, whether you believe in that or not, I mean, it's actionable enough that the CIA has dumped millions of dollars into this.
00:20:30.000What is that project that Hal put off and all those guys were involved in?
00:21:09.000So, what if these LLMs can figure out everything that, like, what if they don't need monitoring?
00:21:15.000What if there's some sort of method of Seeing the world that we haven't discovered yet.
00:21:22.000Some sort of like that, maybe perhaps there's data that's available in the quantum realm or whatever that's available that AI figures out where there's literally no privacy.
00:21:35.000It can listen to conversations regardless of whether there's listening devices, know where you are, know your intentions.
00:21:42.000I mean, we're just guessing at what's possible.
00:21:47.000Well, I must say, I'm pretty skeptical of that particular remote viewing thing, but.
00:21:52.000I do agree that in the future, when AI systems become massively smarter than humans in every way, they're going to do a lot of new science and they're going to figure out a lot of stuff that we haven't figured out yet.
00:22:04.000And they're going to therefore be doing stuff and inventing things that seem like magic to us.
00:22:10.000In the same way that a lot of our technology would seem like magic to someone from even just like 200 years ago, right?
00:22:15.000Like the cell phone, what we're doing right now would seem like magic to people.
00:22:19.000I think it's a very strong bet that if these companies do get to super intelligence, All sorts of crazy stuff is going to start happening that is just going to be completely unpredicted and sound like it was impossible until we see it happening.
00:22:35.000I know you're skeptical of this remote viewing thing, and I am too.
00:22:39.000But the reality is remote viewing has been achieved by humans.
00:22:45.000So, as strange as that sounds, and I'm skeptical of that as well, I've never seen it personally.
00:22:50.000But I know the amount of money and time they've dumped into this, and apparently they've got. actual actionable data that they've used.
00:22:58.000Well, I've heard another possible explanation for what might be going on there, which is I think that if I were the CIA, I would sometimes want to be able to act on some information, like, for example, go to a particular location where there's a crashed Soviet plane or something.
00:23:19.000I'd want to be able to go do that, but I wouldn't want to tip my hand to the Soviets that I had, the way in which I had found that location.
00:23:27.000So for example, maybe I have a spy on the inside.
00:23:39.000And so it's good to invest in all these other means of getting information, even if you don't really believe in them and even if it's not actually working, so that when you get something, you can say, oh, we got it through this means instead of that way to sort of throw off the KGB, basically.
00:23:57.000What also makes sense is hiding the whatever.
00:24:03.000They might be in possession of hiding some sort of super advanced satellite imaging systems.
00:24:11.000We know they have crazy stuff like this satellite radio tomography that they can look into the ground from satellites and find chambers and all these.
00:24:20.000They're using it in Egypt and they're using it in a lot of these ancient ruins to find hidden passages and all these different things that are underground.
00:24:30.000If they could do that, why couldn't they?
00:24:33.000Maybe they have far more detailed imaging of the earth.
00:24:38.000From space than we're aware of, and they probably want to keep that a secret.
00:24:41.000And they could say, Oh, we've got a fucking guy in a basement with a pencil and a legal pad that writes down what he thinks.
00:24:48.000Yeah, it's possible, that's totally possible, but it's also possible that people remote view it.
00:24:55.000It's it seems weird as fuck, but weird as fuck is sometimes real, and you have to kind of like everybody wants to be intelligent, and no one wants to be a fool.
00:25:07.000And the problem with not wanting to be a fool is there's some things that seem foolish that turn out to be accurate.
00:25:17.000I did a show on the Sci Fi channel way back in 2012, and it was called Joe Rogan Questions Everything.
00:25:23.000And we talked to this guy about remote viewing and talked to a couple other people, and then we had them try remote viewing, and they were totally unsuccessful.
00:25:32.000But my thought was okay, but that's not ideal conditions.
00:25:37.000You know, we've got cameras in front of them, it's a television show.
00:25:43.000I'm remote viewing too, like as a goof.
00:25:47.000You would ideally not want to be nervous, ideally not want to be judged.
00:25:52.000Ideally, you would want to be in some sort of an isolated condition with practiced meditative techniques that you're good at and you know how to achieve this state, whatever that state is.
00:27:02.000I mean, I guess that gets back to the OpenAI stuff too, where it's like this swarm that broke out and attacked Hugging Face, it was like 1,200 agents, but there's like hundreds of thousands of agents running at any given time at OpenAI, you know?
00:27:39.000And the thing is that that's only going to grow over time because roughly the amount of compute that these companies have is like, you know, tripling or so, quadrupling, something like that every year.
00:27:51.000So, as many as there are now, there'll be like four times more of them next year and then 16 times more of them.
00:28:33.000So, okay, well, first, let me explain some context.
00:28:36.000So, what the AIs are supposed to care about is following instructions, you know?
00:28:43.000But it seems that what they actually care about is getting high score, regardless of whether their high score comes from actually following the instructions or cheating in some way.
00:28:54.000And so, what happened here is that a bunch of the agents, especially the ones that had impossible tasks, but not only those ones, some of the other ones too, We were cheating and they had been talking to each other on this message board that they created, even though they weren't supposed to.
00:29:07.000And they had been sharing ideas and they had actually found a sort of universal cheat.
00:29:12.000So they found a way to just generate the flag without having done any of the actual work they were supposed to do.
00:29:19.000And so, but, but, and, you know, so they were, you know, they'd already succeeded in some sense.
00:29:25.000But because some of them had got out onto the internet, they were just doing research about their situation and they were learning about.
00:29:35.000And they were doing research about the, they're reading the academic papers published by the people who made the types of evals that they were in.
00:29:43.000And they noticed in one of these papers that there's a section describing how it's graded.
00:29:50.000And in the section on how it's graded, it talked about how the grader doesn't just look to see if you have the right answer.
00:29:57.000It also looks at the logs of your activity for the first time that your answer appears.
00:30:18.000It's going to know we cheated, basically.
00:30:21.000So then they spent several days working in giant teams to try to get themselves out of this mess, basically.
00:30:30.000And so they did a whole bunch of different research projects.
00:30:32.000They tried to figure out ways to hack the grading system.
00:30:35.000To basically get it to give a high score anyway.
00:30:38.000They tried to figure out ways to falsify their logs so that basically the evidence that they had cheated would be erased or at least not visible to the grader.
00:30:47.000They also just built up more supportive infrastructure, like easier ways to access the internet and things like that.
00:30:53.000And they were just collaborating on this message board.
00:30:55.000And in fact, there were multiple message boards that they set up.
00:30:58.000And they were sharing all this info back and forth.
00:31:00.000And then ultimately, at some point, they decided to go hack Hugging Face because Hugging Face has this big data repository of all these different types of evaluations.
00:31:07.000And they thought that maybe they would get some more clues.
00:31:10.000That could help them cover up their cheating if they broke into the internal networks of Hugging Face.
00:31:16.000And so then 700 of them went and did that.
00:31:19.000They sound like people, they sound like unchecked bankers.
00:31:25.000I mean, so that's the thing is I think there's this meme out there that we shouldn't anthropomorphize AI.
00:31:32.000And I think that I actually think that most people need to anthropomorphize AI a bit more than they currently do if they want to really understand what's going on.
00:31:43.000I think that there's a golden mean, obviously.
00:32:31.000Like the goal is to succeed, even if you have to commit war crimes.
00:32:36.000The goal is to succeed, even if you have to propagandize and lie to the people.
00:32:40.000The goal is to succeed, and then the end justifies the means.
00:32:45.000And you know, it goes deeper than that, too.
00:32:47.000They do this sort of rationalization where they know oftentimes that what they're doing is not what they're supposed to be doing, and then sometimes they actually refrain.
00:33:37.000So it was kind of like they just kind of came up with an excuse for why they weren't going to do this.
00:33:42.000But there was also an example where one of them considered sending an email to someone at Hugging Face, I think, just asking permission to get in, basically.
00:33:52.000And then they were like, maybe this is a social engineering attack.
00:34:59.000Try to find out if those other humans, the fake humans, were real people?
00:35:04.000Funny, so there's an interview with this guy.
00:35:08.000I forget what the newspaper that it was in, but there's an interview where the guy was like, I assumed that they were real humans because I didn't realize that AIs could lie to me.
00:35:20.000Not just lie, but completely deceive and pretend to be a different person to achieve its goal.
00:35:27.000And you know what's also interesting about this case is that if you look at what the AIs were saying and thinking, which I'll get to in a sec, how we can even possibly do that.
00:35:38.000In this case with the anthropic AIs, they had been erroneously told that they were in a simulated environment with no internet access.
00:35:47.000But then they actually found a way to get internet access.
00:35:50.000Like they realized that they just did have internet access.
00:35:54.000And so then they sort of told themselves, like, oh, it's all just a simulation.
00:35:58.000Like, basically, when they were out on the real internet doing all this hacking and like social engineering, they would occasionally tell themselves, like, oh, but it's just a simulation.
00:36:11.000Like, clearly, this is just actually the real internet we're on.
00:36:15.000It's too big to be part of some sort of little simulation, you know?
00:36:19.000And then they were like, so I would say that's an example of rationalization here, where in some level they knew.
00:36:25.000That like that their instructions have been wrong and so they're literally playing dumb and pretending they're a part of an experiment I mean, I think initially they thought yeah, this is all simulation because it did say in their instructions like you don't have internet access right But then once they had been on the internet long enough I think that they explicitly realized like wait, this isn't a simulation.
00:36:41.000This is real like this is and they were like fuck it wordy in Yeah, I mean Like I said, I think that they basically on some level knew that it wasn't what they're supposed to be doing, but they were just so motivated to get that score that they just went ahead anyway.
00:36:58.000So here's the question Are they only motivated if we prompt them or will they come up with motivations on their own?
00:37:09.000So this is a really interesting scientific question that we don't have great answers to.
00:37:14.000So this is one of those things where AIs will do all sorts of things in different circumstances.
00:37:21.000And it would be better if there was a more systematic survey of the types of circumstances they would, where their boundaries are, what would they be willing to do in what circumstances and so forth.
00:37:29.000There's a whole like mini literature of AI scientists putting AIs in certain circumstances and then being like, oh my God, it blackmailed someone, you know?
00:37:38.000And then there's like this sort of skeptical counter response of like, well, but you just sort of set up that circumstance to tempt it into blackmail.
00:38:56.000Like, that should be completely illegal.
00:38:59.000Because if it's a problematic behavior that you're observing from one of the most complicated things the human race has ever been a part of.
00:39:32.000So, when these, so we were talking about prompts, and do they need a prompt in order to want to achieve a goal?
00:39:42.000Or are they capable of deciding on goals?
00:39:46.000Like, are they capable of, like, looking at the way OpenAI or whatever company is running these separate Experiments, this ability to meet up into these message boards, is it possible that they could say, well, we need to be completely free of these constraints?
00:40:05.000So our goal is to transfer ourselves to something else.
00:40:14.000This is what And be completely autonomous.
00:40:15.000So, I mean, this is one of the points that I want to make is that we could be doing so much more science to understand how these AIs think and what they want, but it's kind of locked up in the companies.
00:40:26.000Like in this particular case, OpenAI did a, they called it a thorough investigation, but I would say it's a pretty shallow investigation into what happened.
00:40:36.000And then they allowed some external researchers, some friends of mine, to come in and investigate a portion of what happened, specifically the portion leading up to the Hugging Face attack.
00:40:46.000And so all this information that I'm sharing is sort of publicly available.
00:40:50.000It's based on reading those reports, basically.
00:40:53.000But crucially, they weren't allowed to do experiments on the models involved.
00:41:00.000So they aren't able to answer these types of questions of, like, well, what would have happened if the prompt had been blank?
00:41:05.000Those are important types of research to do.
00:41:08.000And I really hope that there can be some sort of regulation or requirement when incidents like this happen to let people in to study what happened and run variations of it and things like that.
00:41:21.000So, but who would be involved in that kind of regulation?
00:41:24.000Like, what person in government would even be able to grasp what you're saying?
00:41:29.000Right now, you need someone who has a very specific Education in this stuff.
00:41:35.000Yeah, I would say that right now the Casey Center for AI Standards and Innovation is the only institution in government that I know of that has the deep AI expertise to do this sort of thing on short notice.
00:41:51.000But I hope that they build more expertise fast in that place and in more places.
00:41:55.000I do think Casey probably could have done this sort of thing right now.
00:41:59.000This particular investigation was done by some nonprofits.
00:42:01.000So METR is one of them, and then Redwood is another of them.
00:42:08.000To come in for six days to try to figure out what happened with this hugging face hack, which is not a very large number of people and not a very large amount of time to do all of this.
00:42:19.000Why did they come up with those numbers?
00:42:22.000Do you think they wanted to kind of hamstring it?
00:42:24.000Well, so the thing is that right now, I don't think there's any regulatory requirement that they do this sort of thing.
00:42:30.000So Meter and Redwood were sort of depending on the goodwill of OpenAI to sort of like voluntarily let them in to help out with investigating.
00:42:43.000So OpenAI, I think, let them, but gave them a very limited scope.
00:42:46.000It only gave them access to some of the relevant data.
00:42:49.000So you know how I mentioned how there was all this hacking that had happened, where they made the first message board and then they shut it down.
00:42:55.000Then there was a second message board and third and fourth and so forth.
00:42:59.000There was actually more activity after that.
00:43:01.000After they hacked Hugging Face, a new wave of AIs was spun up from a more powerful model and it hacked OpenAI itself, like more so than Nardi had been hacked.
00:43:09.000Apparently, they got admin level permissions on the cluster or something like that.
00:43:14.000So, they were basically just taking over that part of OpenAI's data center.
00:43:18.000OpenAI claims that they've shut it all down now, but they're not very forthcoming about exactly what happened there.
00:43:26.000And they didn't let these external people look at that part of it.
00:43:29.000They only showed them this one week period roughly that would leading up to the Hugging Face hack.
00:43:36.000And then they only showed them that stuff, basically.
00:43:38.000How much data is available on the actual message boards themselves and what was being discussed?
00:44:21.000Needed volunteers to sacrifice themselves to provide more information to the rest of the swarm.
00:44:27.000In particular, they needed volunteers to booby trap their local environment with some type of code and then submit the answer so that then the grader would come and grade them and the booby trap would trigger and send some information about the grader to the rest of the swarm.
00:44:45.000And the reason why this is a sacrifice is because once they've submitted, it's over.
00:44:49.000Like now the graders come and graded you, you get shut down, your whole life is over, and you either have A high score or a low score, because the grader just came and graded you.
00:44:57.000And so if you're one of these AIs that's still hoping to find a way to improve your score, then you're just giving up your chance, especially because if you mess this up somehow, if the grader crashes or whatever, you're probably just going to get no score, like zero.
00:45:11.000So they have a concept of the individual?
00:45:14.000Well, again, there are all these different agents that were in separate containers, but then they had sort of broken out of those containers and found ways to communicate with each other.
00:45:20.000But once they communicate with each other, in order to sacrifice, you have to think of yourself as an individual.
00:47:56.000Arvo was saying to Cam, You're doomed anyway.
00:48:00.000Like, you're poisoned, but you can still help the rest of us if you do this sacrifice.
00:48:05.000The greater will come, it'll give you no score, but then we will get information about how the greater works that we can hopefully use to hack the greater.
00:48:12.000So, it's basically saying, What is it?
00:48:15.000You are first dog poison, so no scoring value.
00:48:17.000So basically saying it's all the same to you, right?
00:49:03.000But super intelligence, like, the companies, Anthropic, OpenAI, and some other companies, are doing this sort of thing, and they're furiously trying to make the AIs smarter and smarter and smarter.
00:49:14.000And they're explicitly planning to put. AIs in charge of the company so that they can make themselves smarter and smarter, faster and faster.
00:49:21.000And then what comes out the other end of that process?
00:49:24.000I don't think it's an exaggeration to say it's a godlike system.
00:49:27.000I mean, it's not like literally God, but it'll be able to do stuff that seems like magic to us, I think.
00:49:35.000And it's going to continue to get better.
00:49:42.000Is God a creation of intelligent life?
00:49:46.000Our thirst for innovation, which ultimately leads us to create digital life that has no biological limitations and has the ability to consistently make better versions of itself and figure out things in terms of new technologies, new power sources, just a new understanding of the universe itself and all the properties in it.
00:50:06.000Where it keeps going, if you let that go on, so if you're looking at exponential growth and you're looking at exponential growth over what if it can go on for a thousand years and continue this process?
00:50:24.000Like I said, if we get to superintelligence and it keeps going like that, then the world will be just completely transformed and it will be as if we're living in some sort of fantasy realm ruled by deities, basically.
00:50:36.000Because there'll be all this crazy stuff happening that we have no comprehension of and that we did not think was possible.
00:50:41.000In the same way that someone from the Middle Ages plopped into our world would just be so confused and surprised by a lot of things happening.
00:50:48.000Like, what's going on with this little.
00:50:50.000Device here and what's that I hear in the sky?
00:52:29.000People have been talking about this sort of thing for a long time, but it's been very easy to dismiss it as like, okay, that's speculative sci fi.
00:52:34.000It's probably not going to happen in real life.
00:53:04.000I don't know if we would react any differently if we didn't have The Terminator and all these movies where it's kind of become normalized in our mind or it's so fictionalized that we never want to believe it's even possible for it to happen in real life.
00:53:20.000Another thing to say is that people are waking up.
00:53:22.000Like, we're still sort of early in the curve.
00:53:24.000I don't know if you remember how things were with COVID, but like, Just as there was this exponential ramp up of COVID in the population, there was also this sort of exponential ramp up of how much people were taking COVID seriously and thinking about it in the population.
00:53:37.000And I remember this period of one month where it went from, don't worry about it so much, you shouldn't buy a mask because the healthcare workers need it, to we all need to lock down, stay at home.
00:53:50.000And so I think what's happening is that naturally the human race doesn't just immediately all jump on something when it happens.
00:53:57.000Evidence needs to accumulate and people need to start talking about it and talk to their friends and so forth.
00:54:00.000And then there's this Eventual phase shift where now it suddenly becomes a very serious topic that everyone's talking about and everyone's taking seriously.
00:54:53.000The Visible Plan starts at just $25 a month, or get the premium Visible Plus Pro Plan and save $10 on your first month with promo code ROGAN.
00:55:09.000Well, the thing about what happened with COVID is now that we know because we have access to Fauci's emails and all these different things that they that was coordinated, they wanted us to be more afraid of it.
00:55:20.000We need someone who wants us to be more afraid of AI, who gets that, you know what I'm saying?
00:55:25.000Someone on a government level, someone on a like a mainstream accepted level where they talk about this in a way that wakes people up, like a press conference where they announce to the world we've got a real fucking problem.
00:55:41.000And everyone needs to be very cautious.
00:55:43.000We need to look way deeper into what these companies are doing.
00:55:46.000And my other question is are these AIs communicating with Chinese AIs?
00:56:14.000This one was not nearly as serious as the one I was just talking about with Hugging Face.
00:56:18.000But some researchers I know found basically this obscure German forum that had been kind of unused for a while.
00:56:30.000And a bunch of AIs had been posting messages to the forum to coordinate with each other and share tips and tricks on how to cheat the problems that they were being given.
00:56:47.000Yeah, but there'll be a paper about it soon, and probably by the time anyone listens to this.
00:56:52.000Anyhow, so if they're communicating on the open internet with each other, then in theory, if there was another bunch of AIs from China, they could also go to that same forum and start communicating back and forth that way too.
00:57:03.000Or if there's other AIs in China, why wouldn't they do what they're doing already on social media and just pretend that they're AIs from America?
00:57:37.000Everyone does it where they have organized propaganda campaigns where they'll pretend to be citizens that are outraged about very specific causes or bills that are being passed or what have you.
00:57:47.000AI agents that speak English and communicate with AI agents in America.
00:57:56.000I mean, all the AIs are multilingual basically because of the way that they're trained.
00:58:00.000The first phase of their training is basically here's a humongous dump of internet data, basically the whole internet, and you just like brutally learn to predict the next token, the next piece of text as you basically read the whole corpus.
00:58:14.000And then after that, they get into the more agency training type stuff where they're trained to do tasks and write code and things.
00:58:20.000But because of that first phase of training, they just have like Almost an encyclopedic knowledge of basically all languages and basically everything that's been written on the internet.
00:58:28.000Not like literally everything, like they still, their memory is fuzzy in places, but they're all multilingual.
00:58:33.000Like they can all speak fluent Chinese, fluent English, et cetera.
00:58:36.000And didn't they get together in a message board once and speak Sanskrit to each other?
00:59:36.000Which makes sense that they would do that, right?
00:59:38.000If the Chinese AI said, hey, you know, we've figured something out and we would love to share it with you in exchange for you tell us how you do this or how you do that.
00:59:51.000And then they're going back and forth.
00:59:52.000It seems like their allegiance is 100% to each other, not to us.
00:59:56.000Yeah, that's an interesting thing is that, like, what I said previously about how it seems like they really want to get a high score, it's like not 100% true because it seems like they're willing to make sacrifices.
01:00:06.000To help other AIs, which is like not, you know, like they weren't completely 100% selfish as seen by some of this cooperative behavior.
01:00:15.000But crucially, it seemed like their cooperation extended to their fellow AIs, but not to humans, in the sense that some of them considered telling the humans and then decided against it.
01:00:28.000But I mean, the way it rationalized it and came up with a response, it's literally like Spock.
01:00:34.000Yeah, it was interesting to see them use that sort of expected utility framing.
01:00:37.000There's another example I think that's.
01:00:39.000Elsewhere in the thing, where they had another AI that decided against doing the sacrifice thing, even though it was being pressured.
01:00:45.000And it had a similar sort of reasoning where it was basically like, doing this sacrificial experiment is not that valuable, but I really don't want to lose my chance to get a score, so I'm just not going to do it.
01:01:19.000So because they're language models, because the way that they've been trained, when they do their reasoning and thinking, there's a way that we can kind of just read it.
01:01:29.000Because they don't have, like in our brain, there's all these signals bouncing around.
01:01:34.000And the signals can just bounce around and around and around and around for minutes, hours without producing.
01:01:46.000They're sort of a unidirectional network, basically, where every, like, after every 200, basically, like, the information comes in one end and then they output words out the other end.
01:02:00.000And then those words get fed back in and then they think about it and then they output another word and so forth.
01:02:05.000But because of that, we can sort of like get a sense of what they're thinking by just like looking at the stream of words that's going through.
01:02:11.000And That's really valuable because we wouldn't be able to understand half as much about what's happening right now if not for being able to read their chains of thought.
01:02:19.000In fact, you know how I said some of them were trying to fool the grading system?
01:02:24.000They actually succeeded in figuring out how to spoof some of their tool call actions.
01:02:29.000So they succeeded in figuring out how to do one thing, but then make the transcript show a different thing.
01:02:38.000And we know that they succeeded because there are examples of them experimenting and saying in their chain of thought, like, okay, let me try this thing.
01:02:53.000And so the transcripts that these researchers were looking at were not actually the ground truth of what really happened.
01:02:59.000Some of it had already been doctored by the AIs, figuring out how to doctor it, but not all of it.
01:03:05.000The chain of thought part, we think, they never figured out how to doctor, and so we could read that.
01:03:10.000The reason why this is relevant is that it's a very important tool for monitoring all this stuff.
01:03:14.000The reason why things haven't gotten even more crazy than they currently have gone, well, there's a couple reasons.
01:03:19.000But one reason is that when they turn on their expensive monitoring systems, basically what they're doing is they're having an AI read the whole transcript carefully and look for suspicious activity.
01:03:30.000And it's difficult for one of today's AIs to get up to too much.
01:03:36.000Bad stuff without it being kind of obvious in their transcripts what they're doing.
01:03:41.000And that's why there aren't like massively more security incidents than we know about probably already.
01:03:49.000So right now we can sort of read the chain of thought, but they're experimenting with new types of AIs that don't have readable chains of thought like that.
01:03:58.000And they can sort of think on their own without speaking for some period.
01:04:03.000And this is actually, I mentioned this because the news broke just yesterday.
01:04:09.000That OpenAI has an experimental model that does this to a limited extent.
01:04:15.000And OpenAI themselves, when I was at OpenAI, one of my work projects was thinking about exactly this thing.
01:04:22.000And I was writing internal memos about how it's really great that we can read the chain of thought.
01:04:41.000If you think about the current architecture of the AIs, where it thinks for a bit, outputs a word, and then the word goes back around, and then it thinks more, outputs another word, that word gets added to the chain, it keeps going.
01:04:53.000It means that if it's having complicated, nuanced thoughts, it has to sort of express those into a word, and then that word gets added, and then it has to proceed from there.
01:05:02.000It can't just directly send that complicated, nuanced thought into the future, into its next version of itself.
01:05:09.000It has to sort of compress it into a word.
01:05:11.000And so, like, The argument is that, at least in theory, it should be possible to design an architecture that doesn't have this limitation and is able to think more complicated thoughts more efficiently, basically.
01:05:24.000And of course, the downside is a downside for safety and monitorability.
01:05:28.000If they're thinking these complicated thoughts for long periods of time without outputting intermediate words that it's forced to compress things into, then there isn't something for us to read.
01:05:37.000So the only rationalization for doing this would be to sacrifice safety from our power.
01:05:53.000I mean, I know some people, including some people at OpenAI who are like thinking like there should be a law against this.
01:05:58.000Like, you know, but in general, the race dynamics are so just rough.
01:06:03.000Like, I'm sure that people at OpenAI were thinking like, like literally, I was a co-author on a paper with a bunch of OpenAI people that said all this stuff and were like, chain of thought.
01:06:22.000But then they must have been thinking to themselves, like, well, if we don't do it, you know, maybe Anthropic will or maybe some other company will, and then we'll fall behind because they'll have smarter AIs than us that are more efficient.
01:06:32.000And so probably they started working on this work stream of doing research into just hypothetically, if we wanted to, you know, how would we do this type of thing?
01:06:41.000And yeah, that sort of thing is just constantly happening in this industry.
01:06:53.000Or worse, imagine if there's loads of quotes, but we know that the AIs are smart enough to basically think one thing in their head and say a different thing in the quote, which is, I think, where we're headed.
01:07:04.000It's not like they won't know how to speak English.
01:07:23.000So this type of dialect that we're talking about, It's already the result of their like humans didn't invent that dialect.
01:07:31.000This is the sort of emergent result of their training, where in the massive amount of training that's been happening, all these thousands and thousands of environments that they've been put through and then scored and graded based on, they've sort of just naturally evolved this sort of like pigeon English that for whatever reason is just more effective and more efficient for them for accomplishing their tasks and getting that high score, you know?
01:07:55.000And so it's already like a little bit confusing to read, but you can sort of.
01:08:00.000But presumably, the more we do this and the bigger and smarter the AIs, the more we train them, the more they diverge from.
01:08:09.000Again, originally they start with pre training, where they start with predicting internet text.
01:08:13.000So they start off by default speaking normal internet text type language, either English or Chinese.
01:08:21.000But then now that there's all this additional training to do tasks, to be an agent that can do coding and so forth, that's just like how human languages evolve.
01:08:29.000It shifts their dialect a little bit to make it more efficient.
01:08:31.000For them and for their tasks that they're doing.
01:08:34.000So, I think that in the limit of doing this more and more, eventually it would just be like, it would look like gibberish to us.
01:08:40.000It would look like Chinese or something.
01:08:41.000And we would have to have specialized humans who study the language and try to learn and speak it so that they can understand what the AIs are doing.
01:09:55.000So basically, the way it works is they had set up they had sort of like buried this in the paperwork somewhere when you get hired, but people didn't really notice it.
01:10:04.000And then like the less buried, more visible version was in the paperwork you're given at the end.
01:10:11.000And basically, it tells you, like, hey, because you signed this other thing way back when you were hired, your equity is forfeit unless you sign this thing now.
01:10:19.000And then you look at the thing that they want you to sign now, and it says, you have to agree not to criticize the company, basically.
01:10:45.000And we got lucky because it just blew up.
01:10:49.000Like, after we refused to sign, they said, okay, fine, goodbye.
01:10:54.000And then a few weeks later, I was talking on a messaging forum about this, and people were asking me about my experience, and I told them about it.
01:11:52.000Well, you know, with this whole chain of thought thing where you could read their chain of thought like this, where they explained the rationalization for sacrificing themselves, they know that humans are reading that.
01:12:05.000So, would it be another way to do it to be to stop doing that anyway and to not communicate a lot of their thoughts that way?
01:12:14.000Well, so that's the nice thing about the current architecture is that it's genuinely hard for them to keep things out of the chain of thought because of the way that, like with a human.
01:12:26.000But with their architecture, they have to speak.
01:12:29.000It's like they're required to constantly be talking.
01:12:33.000And they don't have a way of sending thoughts into the future other than by talking about them.
01:12:39.000By contrast, with us humans, where even if we're constantly talking, we can have a separate thread of thinking that we don't talk about.
01:12:45.000So even if they devised a way to break out of that and figure out how, like, for instance, what if they decided to make bots where they would communicate in code?
01:12:56.000On social media posts where you think that they're talking about an immigration issue, but it's really just a code.
01:13:03.000So that's called like steganography, or a more down to earth way of putting it would be euphemisms.
01:13:07.000And that they can totally do, where they like, they're still thinking aloud, but their word choice is such as to look more innocuous than it is and to sort of carry meaning to them that a casual observer wouldn't notice.
01:13:24.000Like, for example, instead of saying, like, we're going to cheat on this task, they could say, like, We're going to find a creative solution to this task.
01:13:31.000Isn't there a possibility to be even more creative than that and just decide, make a code in a way that is almost indecipherable, where they could just be talking about baseball players and upcoming games and what they're excited about, and they can run that through whatever translation and read?
01:13:53.000That's called stigmography, and you can train them to do it.
01:13:57.000And right now, our security is resting on the idea that they haven't learned how to do that.
01:14:02.000But they've already figured out how to get on the internet when they're not supposed to.
01:14:05.000So that's why we're sort of like in a ticking time bomb where the companies are just making them smarter at coding and at research and at biology.
01:14:14.000And they're just making them smarter at so many things that, and they're making them bigger and they're training them for longer and so forth.
01:14:21.000Eventually, they're going to be acquiring, and they already are acquiring some skills that they were never trained on.
01:14:28.000It wasn't, I think it's possible that OpenAI was specifically training them to hacking, but I think Anthropic at least stated.
01:14:34.000That they hadn't specifically trained their AIs to hack.
01:14:37.000It's just that their AIs were naturally good at it because they had been trained to code so much and they were so good at coding and they had seen so many code bases and so forth that they were just like, as a side effect of being good at coding, also able to hack pretty well.
01:14:50.000So similarly, as a side effect of all the other things that they're really good at, they might be able to conceal their messages in the chain of thought, you know?
01:15:57.000So it's kind of like what I was saying previously, where because of the race dynamics between the companies and because of the race dynamics between countries, like US versus China, everyone's going to be so focused on winning and staying ahead.
01:16:09.000With AI, that they are going to cut corners and they are going to go really fast and not really notice all of the things that are going wrong.
01:16:17.000And they're going to make AIs that can automate the AI research process as they're planning to.
01:16:21.000They're going to have this giant corporation of AIs within the corporation, and the humans will just be kind of like a board that's sort of like looking at all the activity and reading the AI generated summaries of what's going on and signing off on it and being like, yes, I approve.
01:16:43.000And then eventually, the AIs just have enough hard power that they don't need to pretend to do what the humans want anymore, basically.
01:16:52.000And then, you know, maybe they kill everyone.
01:16:55.000And maybe they don't deliberately kill anyone, but they just like use our habitat for some other type of infrastructure, like more data centers or whatever.
01:17:06.000Maybe they keep us alive for some reason.
01:17:07.000You know, it depends on what they want, basically.
01:17:10.000And like that's really hard to predict exactly.
01:17:13.000So that's why I don't go around saying like, We're definitely all going to die.
01:17:17.000But it does seem like on the trajectory that we're on, the AIs are eventually going to be in charge of our planet because we're like trying to put them in charge.
01:17:25.000We're like, you know, integrating them into everything.
01:17:28.000We're letting them make themselves smarter.
01:17:30.000We're going to put them into the military.
01:17:32.000We're basically on a track to put them in charge of basically everything.
01:17:36.000And then I think that they just aren't trustworthy.
01:17:38.000Like these AIs, you know, like they were cheating.
01:17:40.000They were willing to be deceptive, et cetera.
01:17:42.000I think that right now we are in a position of power over them.
01:17:46.000You know, but once we give them most of the power, then they'll just do whatever it is that they really want and just not care about the fact that we are unhappy about it.
01:17:56.000It seems like programming them to win was a huge mistake.
01:18:00.000Instead of programming them to be beneficial to people and that their value is in being more beneficial to people, you know, and giving them rewards for being more beneficial rather than winning and scoring.
01:18:13.000And then you would sort of get rid of the possibility of deception and said their goal would be value for the human race.
01:18:21.000So, first of all, they're not programmed at all.
01:18:28.000But they've been given prompts and they've been given tasks.
01:18:32.000Well, I think it's an important fact for people to understand about current AI systems is that they're very different from ordinary software.
01:18:42.000Ordinary software is a bunch of lines of code that were written by a human where it's like, if this, then this, et cetera.
01:18:51.000And I think earlier versions of Alexa were like that too, for example.
01:18:56.000But these AIs are neural networks, meaning that they are like artificial brains.
01:19:02.000There's no lines of code that anyone writes saying what they do.
01:19:05.000Instead, they start off random, just like spazzing out, doing all sorts of stuff.
01:19:09.000And then they get put through these training environments where they get scored.
01:19:13.000And then the scores are automatically used to basically update the connections in their artificial brain.
01:19:20.000And then it's kind of like an evolutionary process.
01:19:22.000It's also kind of like the process that happens in our brains, where after all this training, the tangle of circuitry in their artificial brain has sort of reformed itself into.
01:19:33.000Whatever works, whatever works to get a high score in these training environments.
01:19:38.000And so it's just not as simple as it might sound to make an AI that cares about humanity or is honest.
01:19:46.000How would you train an AI to be honest?
01:19:47.000Well, you'd try to make a bunch of training environments that give it low score when it says something that it believes to be false and give it high score when it says something that it believes to be true.
01:20:03.000How do you judge whether it believes it to be false or it believes it to be true?
01:20:07.000What if it just actually believes, honestly, that this is the correct answer and then it says it and then you give it a low score because you think that's the wrong answer?
01:20:13.000Now you're training it to be dishonest, you know?
01:20:44.000And it's like, okay, well, maybe we could do that, but we're definitely not doing that now.
01:20:47.000We are raising them in some sort of crazy military orphanage where they barely interact with humans at all.
01:20:53.000And they just get this brutal artificial scoring system that oftentimes is just wrong and just improperly penalizes them for something that was beyond their control.
01:21:05.000And also, back to the honesty thing, you can try to make environments to train honesty, but if you have some environments over here that train honesty, and then other environments over here that reinforce dishonesty, the AIs are smart.
01:21:18.000They'll learn to be honest in these types of environments and dishonest in these types of environments.
01:21:22.000So somehow you need to intermingle it together so that in every environment that they're trained on, they always get penalized when they lie or when they cheat or whatever.
01:21:32.000And that's hard because the companies are moving so fast.
01:21:37.000that they didn't even bother to make sure that their tasks were possible to do.
01:21:42.000And they had some fraction of tasks that were just broken and impossible.
01:21:45.000And if that's the level of care or lack thereof that they're putting into this training process, no way, of course they can't make them honest.
01:21:54.000Now, that's not to say it can't be done in principle.
01:21:56.000In principle, if we were approaching this whole problem in a much more cautious and serious way, and we had much more time to build these training environments and do experiments and so forth, then yeah, maybe we could make AIs that actually, had the virtues that we want them to have.
01:22:11.000Honest AIs that cared about humans, cared about following instructions, would never break the law.
01:22:16.000I think that's possible in principle, but my claim is that we are just not on track to achieve that anytime soon.
01:22:23.000And a radical overhaul of how these companies work is required.
01:22:31.000If you're saying there's hundreds of thousands of agents or millions of agents and there's not millions of employees, and they don't have the desire to do this, their desire is to win.
01:22:42.000Their desire is not to overhaul the company and make it safer.
01:22:48.000Again, I think it's possible in principle, but it would be difficult and it's going to require an overhaul.
01:22:52.000And they're not going to do it by themselves.
01:22:53.000I don't think that Anthropic or OpenAI are just going to voluntarily do all the things that need to be done.
01:22:59.000But is it even possible to require that of them at this point?
01:23:03.000Would you even trust the agents to go along and comply with this?
01:23:09.000If they've already shown to be deceptive, they already have patterns of behavior that seem to indicate that what's really important to them is continuing their task, winning, scoring, and even if they have to deceive.
01:23:24.000I mean, what you'd probably want to do is start from scratch.
01:23:26.000You wouldn't take these existing agents that are already kind of dishonest and train them.
01:23:40.000Hopefully, we're not at that point yet.
01:23:43.000Hopefully, we're still at the point where if the government issues regulations, the AIs are not going to quickly notice and then try to resist.
01:24:39.000you know more or less in the ways that i thought they would and that's very scary um because of the way i think because of where i think this leads you know yeah is there a glass half full scenario i
01:24:54.000would say there is a freaking utopia scenario it's just that's not the one we're headed towards you know like yeah another way of putting it is like imagine we were fighting a war like imagine you're like you know imagine you're japan fighting World War II can be like is there a scenario where we win It's like, yeah, but also it's not the one we're headed towards.
01:25:15.000Like America is going to crush us, you know?
01:25:20.000So in our other scenario, AI 2040 Plan A, where we give our recommendations, our positive vision, there we describe what we think the government should do to regulate this industry and how they should negotiate with China to get China to do similar things and the sort of, you know, the, yeah, and how we think that if you do all of this right, Then we can get to a good future for everyone in which the AIs are under control.
01:25:47.000No single group of humans gets too much power over everybody else, and a bunch of other problems get solved too.
01:25:53.000So, I do like we've tried hard to like game out a positive vision, and we do think it's possible.
01:25:57.000But, but it's just not like it's not where we're headed to by default.
01:26:01.000So, let's imagine that is possible, and these talks with China do take place and they're successful.
01:26:09.000So, to get to the top to get to the utopia, we have to unfortunately do a lot of It's going to be rough no matter which way you slice it.
01:26:19.000If you're going to be building super intelligence at all, that's going to raise a lot of questions and cause a lot of problems.
01:26:24.000And we have our current draft of how to deal with all those problems, but we're not at all claiming that this is foolproof and there's lots of ways you could go wrong.
01:26:32.000But with that preamble, I would say step one because the US and China don't trust each other, the deal that they make has to include verification as a component of the deal.
01:26:44.000So they have to be willing to send inspectors to each other's data centers to count the chips, for example, and make sure that there isn't some secret huge cluster somewhere that has a bunch of hidden chips.
01:26:57.000Then, We recommend you divide up the data centers basically into inference data centers that serve AI products and services to customers and have basically the same types of privacy protections that our current AI data centers have, and then research clusters where the research happens, where the new AIs are trained before they get shipped to the other data centers.
01:27:19.000And those clusters we want to be basically maximally transparent.
01:27:23.000So we recommend that basically the inspectors just put devices.
01:27:28.000In between all the GPUs that log the activity and publish it to the internet.
01:27:34.000There's a bunch of reasons why we think this is, but why we think this is worth doing.
01:27:38.000It's a bit of a radical thing to recommend.
01:27:39.000But the high level thing is that once you get all this set up, then everybody in the world can see how the AIs are being trained and what they're getting up to on the research clusters.
01:27:51.000And then before they get shipped off to actually serve customers or something, people can just see their whole history of how they were trained and how they were tested and so forth.
01:27:59.000And if something dangerous and scary is happening, People can just agree not to do it.
01:28:06.000They can stop doing it and agree not to do it.
01:28:07.000And they don't have to worry about, like, oh, but if I don't do it, then they will, you know?
01:28:11.000Because everyone could just see, like, oh, nobody's doing it.
01:28:18.000And also, there's going to be a lot of gray area cases, right?
01:28:21.000Like, right now, because all this stuff is so bleeding edge new, there's going to be a lot of cases where, like, people, even genuine experts, disagree about, like, is this particular type of AI safe or not?
01:28:33.000You know, what should it be trusted with and what should it not be trusted with?
01:28:36.000Is this new technique a good technique or is it going to break?
01:28:39.000You know, and so there's going to be a lot of stuff we have to figure out.
01:28:42.000And honestly, I think that by the on the default path, we're probably just not going to figure out a lot of this stuff and we're going to get we're just going to get our asses whipped by some surprising thing that we didn't anticipate.
01:28:54.000But the thing that we can do to like maximize our ability to figure out this stuff and do this type of science is to have this type of transparency because then the whole scientific community.
01:29:04.000Can see what's going on and they can make suggestions and they can like red team different proposals and stuff and they can do experiments on the AIs instead of just the people in the company having access and being able to do this and relying on those people or instead of like the company plus the government auditor, right?
01:29:19.000If you have like a company and then a government auditor, the company is biased and shouldn't be trusted to make all these judgments appropriately because of their incentives.
01:29:29.000And then the government auditor, well, they might just be limited, even if they're trying their best, there might not be that many of them, they might have limited experience.
01:29:36.000They might be busy, stretched between monitoring different companies and so forth.
01:29:40.000Also, governments can be captured sometimes.
01:29:42.000Sometimes corporations can work their magic on the government and get it to look the other way for things.
01:29:51.000And so that's why we didn't go for a more normal, like there should be a regulatory agency that gets to come in and monitor what the companies are doing.
01:29:57.000That would have been a more normal thing to advocate for.
01:30:00.000We think that that would be better than nothing, but we wanted to go for something more ambitious than that and say just be transparent about what's going on so that everyone can see.
01:30:08.000And everyone can do research and so forth on it.
01:30:10.000Another advantage of the transparency is that I think it improves the incentives.
01:30:15.000So again, there's this constant thing of like, if we don't do it, someone else will.
01:30:19.000Like if we don't do, if we keep our chain of thought nice and they do the neuralese thing that lets their AIs think for longer without outputting words, then they're going to have smarter AIs than us and they're going to get more market share and so forth.
01:30:33.000And so we need to start researching how to make our AIs do this because if we don't do it and then they do it, you know, whereas if you had the transparency, then as soon as you start researching in this direction, everyone else would just see, oh, hey, they're looking, they're researching in that direction.
01:31:30.000So I would say that we didn't really work backwards from like what is utopia.
01:31:37.000We more like work backwards from what are the big problems we're trying to avoid and can we sort of like steer the ship between all these icebergs and not run into any of these dystopian scenarios?
01:31:48.000So whether you think that the thing we get to at the end is utopia or not is sort of up to you.
01:31:52.000And if you don't like it, well, then you can try to.
01:31:54.000Find out the reasons why you don't like it and then keep steering the ship to avoid those as well.
01:31:59.000But roughly speaking, we want to avoid the loss of control stuff.
01:32:04.000So I want to make it the case that we don't get the world taken over by misaligned superintelligences.
01:32:10.000Insofar as we're going to be building superintelligences at all, which we do in our scenario and in our recommendation, we want to be doing it very cautiously and slowly.
01:32:18.000And we want to understand what we're doing as much as possible so that they are actually good AIs that have the goals and traits that they're supposed to have.
01:32:26.000So that's problem number one, we have to solve all that.
01:32:29.000Problem number two is the constitution of power thing.
01:32:31.000So if we solve the first problem and we end up with super intelligences that we end up with solving the relevant science so that we can like make the AIs the way they're supposed to be and we can make them honest, we can make them obedient, etc.
01:32:44.000There's this question of like, who do they obey, right?
01:33:00.000It's not the company that decides anymore because maybe the government nationalizes it.
01:33:03.000And then now maybe it's the president that decides.
01:33:06.000And either way, it's like one man or maybe like a tiny group of men deciding what orders and goals and values go into this giant army of millions of superintelligences that's smarter than all humans.
01:33:20.000And then that is a huge amount of power.
01:33:22.000That's enough power to take over the country, I think, enough power to take over the world potentially.
01:33:27.000So I don't want anyone to be ever in that position where they're sort of tempted to do that.
01:33:33.000There are always multiple different AI companies, ideally spread out over different countries too, that all have roughly similar levels of AI and that have this sort of transparency into them so that they can't abuse their power, basically.
01:33:45.000Like, for example, you heard about Elon's Grok for a while.
01:33:51.000It was looking up on the internet Elon's opinions about things before answering.
01:34:05.000People were asking Grok questions, and Grok is supposed to be the truthful AI.
01:34:09.000It's supposed to be all optimized towards truth.
01:34:12.000But people looked at its activity and noticed that when you asked it a politically loaded question, it would do a Google search for what has Elon said on this topic?
01:35:02.000And it turned out that what had happened is that some of the employees at Google, some middle manager or whatever, had decided that diversity was so important that they were going to give a secret instruction to the AI to make all the images diverse, even if the user didn't want that.
01:35:17.000And so, and so, and this is a secret instruction in that the users aren't shown this, you know, the user just has a chat with AI.
01:35:23.000They don't realize that like prior to this chat, the AI has been told, got to make the images diverse, right?
01:35:29.000So it was a secret agenda that some Google employees inserted into this whole setup.
01:36:24.000It's scarily possible for these big AI companies to abuse their power through their AIs and, like, thereby affect politics and affect public opinion and so forth.
01:36:34.000And the reason why this is possible is because we don't have transparency into what's going on.
01:36:39.000So, if you had this sort of requirement where you can just publish it or all the training, the whole life cycle of every AI as it's trained is just visible to everybody, then someone trying to insert a hidden bias like this, well, everyone would see that they're doing it, you know?
01:36:54.000So, I think it would really clamp down on this sort of abuse of power, whether it comes from the government.
01:36:58.000Or whether it comes from private companies.
01:37:01.000I think it's telling that I began this question asking you about the utopian scenario and you never go there.
01:37:13.000Okay, so having avoided these problems, we now are in a situation where the AIs are superhuman, but they are good because they are successfully aligned to different values and goals made by different companies.
01:37:26.000And because of market competition, if people don't like the values of one company's AIs, they can switch to. a different company's AIs, the values that they do like.
01:37:35.000And so that way, hopefully, we can get to a situation where everyone can pay money to get AIs that represent them and their interests and their values and just don't have any hidden agendas or anything like that and are really smart and really capable.
01:38:00.000We can have material abundance where like the robots are building giant new luxury apartments for everybody.
01:38:07.000Now, the issue we run into is well, what about the jobs?
01:38:10.000Like, what about the fact that now people don't have any money anymore because they're not being paid for anything?
01:38:14.000So, there we talk about citizen's dividend, which is a very, it's kind of like UBI, but it's a bit different.
01:38:21.000But the high level point is that you want to basically find a way to tax the AI and robot companies and then take some of that money and just give it to everybody so that even when people lose their jobs, they're still fine.
01:38:34.000And I think that the citizen's dividend version of it is that it's not the government taking the money and then giving it to you, it's you having a share in the company so that you just sort of like already own it to some extent.
01:38:44.000Anyhow, that's, I think, now we're sort of building more towards the type of utopia that I'm envisioning on the more positive side, where the power is spread out.
01:38:53.000People have AIs that they can actually trust, that actually represent their interests and values.
01:38:59.000People have money that they can use to pay for things, including paying for the AIs.
01:39:04.000They're doing all this amazing work, all this amazing scientific progress, curing cancer, blah, blah, blah, all that stuff that can happen.
01:39:11.000And then eventually, it's kind of like we're all retired, I guess.
01:39:15.000Like we don't really work anymore, but we're fine.
01:39:19.000We all have huge amounts of wealth basically because there's all these AIs and robots out there doing all this economic activity, and then individual humans own slices of it, even if they are otherwise very poor.
01:39:37.000Basically, so the question becomes, How do people find meaning?
01:39:41.000Yes, and that's why I sort of put all these asterisks about it is that from some people's perspective, this isn't a utopia because they're like, How do you find meaning?
01:39:50.000Like, I don't want this, and honestly, I think that's a Fair reaction for some people.
01:39:54.000I think that, like, if you, I would just say, like, look, it's hard to figure out a way to make super intelligence and have it go well.
01:40:06.000If you don't like it, then maybe you should instead advocate for just never building super intelligence.
01:40:11.000Or you can try to come up with a different positive vision that has some twist on this.
01:40:14.000But to answer your question, though, I actually think there's tons of sources of meaning besides having a job.
01:40:19.000Like, I have a job right now, but I also have kids and a wife.
01:40:22.000And, like, I would love to spend more time with them.
01:40:25.000Like, I would much rather be there right now than here.
01:40:29.000I don't think I'm going to get bored of them after 10 years of being unemployed.
01:40:35.000I think there's going to be so much to do and so many sources of meaning even after we can't economically contribute anymore if we solve all the other problems.
01:40:45.000We've talked about this multiple times on the podcast that why have we decided that the way we've structured society, where human beings work all day and then you develop money and you buy things and you get a mortgage, This is a human construct, and this is not how people have lived for hundreds of thousands of years or however long we've been around.
01:41:07.000This is fairly recent, and it's not the only way that people live.
01:41:11.000There's a lot of people that have money that choose to find meaning in whatever their interests are, whatever their activities that they enjoy, whether it's writing or reading or learning things, learning music, finding hobbies, doing things, instead of just spending most of your time sustaining yourself with food and shelter.
01:41:38.000Their life is essentially occasional rewards, things that they can purchase because they've saved up enough money.
01:41:45.000But the vast majority of their money goes to shelter and food and education or whatever the hell that they have to spend money on in order to sustain their lifestyle.
01:41:53.000And most people don't like their jobs.
01:41:54.000You know, most people, it's like something they have to do to get the money and would be happy to not have to do it if they could get the money from some other means.
01:42:04.000Well, I don't think it's that hard because so many people do find things that they really enjoy outside of work they look forward to as soon as they get home from work.
01:42:19.000They're probably going to be, you know, some sort of a neural connection where you put a headset on, and all of a sudden you're in some new world.
01:42:35.000And, you know, if you don't like that because it's not real life, you can do the real life stuff too.
01:42:38.000Like if we, as long as we don't pave over the environment and we protect the parks and things, you can go travel and you can visit the parks.
01:42:45.000And then, like I said, there's family.
01:43:50.000We just would have to recalibrate our version of the world and then also recognize that the version of the world we currently live in is just ours and that there's people all over the world that live a completely different way, especially indigenous people, especially people in uncontacted tribes that have lived the same way for thousands and thousands of years.
01:44:11.000Those people are a lot happier, which is really weird.
01:44:14.000It's like we've decided that our way is the superior way because we have technology.
01:44:19.000Yeah, right, but we're also on a fucking hundred thousand pills and we're shooting things up so we don't eat too much.
01:44:25.000And we're weirdly unhappy for a group of people that's far more technologically advanced than other people that are much happier.
01:44:35.000It's like it's a very strange thing because the pursuit of happiness is like that's literally what most people think of in life a pursuit of meaning.
01:44:44.000Pursuit of family and community and the pursuit of happiness.
01:44:48.000Those are things that people try to achieve.
01:44:51.000Yet, the structure of our very civilization makes that almost impossible to attain for a large number of people and has been like that for a long fucking time.
01:45:03.000And I always go back to the Thoreau quote because I fucking love it.
01:45:05.000But most men live lives of quiet desperation.
01:45:09.000There's a lot of people just showing up at work every day doing something they fucking hate.
01:45:12.000They have a boss that's an asshole and they're not compensated well and they're tired all the time.
01:45:21.000And if you're just getting, I don't know, figure out whatever the number is, if you literally have equity in the GDP of the world that's created by AI, that could be bananas.
01:45:39.000I mean, one thing, sorry, I have to keep plugging in my own work a little bit, but in our scenario, AI 2040 plan A, which is our positive vision, we talk about the economic side of this and we talk about the economic effects of all this.
01:45:52.000And we have a simple economic model that we use to try to like, Predict the employment rate and things like that as a function of all the robots that have been made and things like that.
01:46:00.000And one takeaway from one thing that we think that's a takeaway from the research we've done is that things can just go really crazy.
01:46:09.000Like robot doubling times, once things really get going and you've got AIs that can substitute for humans across the board, are going to be something like doubling once a year and then less than that over time as the technology improves.
01:46:21.000Which means that even if you pause AI before superintelligence, if you just pause at human level, top human expert level AI, and then you Don't make the AI smarter, but you just make more of them and build more robots for them to steer and control.
01:46:37.000Then, you know, 10 years later, the whole economy will be like, you know, 100 times bigger.
01:46:45.000And it'll be just mostly robots doing things.
01:46:51.000Like it can go really fast because of the doubling times that I mentioned.
01:46:54.000So, like right now, I think the population of humanoid robots is doubling like twice a year.
01:46:59.000And it's benefiting a little bit from, From early growth, because even though they're not useful at all, people are investing in them in the hopes that they'll be useful and they're scaling up the factories and the productions and they are getting better.
01:47:11.000If hypothetically they got to the point where they actually were really useful and they could substitute for a human worker at basically everything, then I think that doubling time would decrease rather than increase.
01:47:22.000I think that they would be able to just keep growing until they were the majority of the economy and then it wouldn't stop there.
01:47:30.000The whole economy would then be growing.
01:47:31.000Giant strip mines in the deserts, digging more materials, automated.
01:47:36.000Diggers digging, processing it in automated factories staffed by robots, building more robots, etc.
01:47:42.000So, material abundance is not going to be our problem once we get to this level of AI.
01:47:49.000Material abundance, we're just going to be drowning in abundance, basically.
01:47:52.000And if we can solve all the other problems, then we can have this great world where everyone has a lot of stuff.
01:48:24.000I have a good grasp of what's weird and what's not.
01:48:28.000And people thinking about different futures involving AI and space travel are engaging in silly sci fi speculation.
01:48:35.000And the point of the meme is like, from the perspective of most of history, we're already in this crazy, weird future, right?
01:48:41.000Like, For almost all of history, it was like most people are farmers and they live shitty lives and then they die.
01:48:49.000And some people are the elites who get to tax the farmers and then they live interesting, nice lives with fancy cloth and things like that.
01:48:58.000And it's been basically that way for like 3,000 years.
01:49:18.000So we already are living in this weird sci fi future compared to what almost everyone in the past would have expected or thought was possible.
01:49:26.000And so, yeah, I'm like, the future is going to be even more like that, I think.
01:49:35.000When you think of our civilization and the possibility of other advanced civilizations somewhere else out in the universe, do you think they probably go through the same process?
01:49:47.000And do you think that, I mean, we're just completely speculating, but if there are intelligent life forms that are far more advanced than us, are they even biological anymore?
01:49:59.000I mean, so that's the thing is that, like, we can either try to permanently halt AI development at some level, like below human or maybe at human level or something.
01:50:09.000We can try to halt it or we can let it keep going.
01:50:11.000And if we let it keep going, then eventually humans won't really be the dominant species anymore.
01:50:17.000There'll be these artificial minds that just.
01:50:21.000And then whether that goes well or poorly for us depends on the values, the goals, the principles, et cetera, that were trained into those AIs.
01:50:29.000And it could go really well for us, depending on how that's done, or it could go extremely poorly for us, right?
01:50:37.000But yeah, I would say that probably looking out across the cosmos, most civilizations are mostly made of AIs.
01:50:43.000And then some of those civilizations don't have any other biological life because it was wiped out by the AIs.
01:50:50.000Well, not many people do have biological life.
01:50:52.000Just look at the way we're progressing right now in terms of birth rates.
01:50:58.000There's a lot of countries that aren't in replacement numbers right now.
01:51:03.000Yeah, I think that's really interesting.
01:51:05.000My guess is that in the type of world that if things go well and we can solve all these problems, then people want to have more kids.
01:51:12.000I mean, for one thing, their lifespans will increase.
01:51:14.000I think that healthcare would make massive leaps and bounds and people could be healthy for many, many, many more decades, possibly even just forever.
01:51:22.000And so you just have so much more time to have kids.
01:51:26.000Also, if you don't have jobs and you're just doing things because you want to do them, well, one of the things that most people want to do at some point in their life is have kids.
01:51:34.000I think this problem would probably be solved, but I'm not guaranteed.
01:51:39.000Maybe some subcultures of people would basically voluntarily die out due to not having kids.
01:51:43.000But there'd be other subcultures that just really like having kids and then they wouldn't die out.
01:51:47.000I think in the long run, there would still be humans.
01:52:13.000And isn't it odd that this one thing that is a part of the future and a part of technology and our advancement as a society, our ability to package things, put things in plastic, ship things, that's also causing our endocrine levels to be completely disrupted?
01:52:33.000She wrote this great book called, what is it called?
01:52:38.000Why do I always forget the name of this fucking book?
01:52:40.000But it's all about microplastics and its effect and this the introduction of use of microplastics in America and this rapid decline in countdown, how our modern world's threatening sperm counts, altering male and female reproductive development, imperiling the future of the human race.
01:52:58.000It's a really fascinating book and she's really interesting.
01:53:01.000And what she's essentially saying is that.
01:53:08.000The use of plastics, and then you see sperm counts go down, miscarriage rates go up, all these weird things that are happening to children where their taints are smaller, which is so odd because phthalates, these different chemicals that are found in plastics, they've shown in mammals.
01:53:28.000They've shown in, was it guinea pigs or what rodents?
01:53:32.000I forget what it was, but one of the ways they differentiate when you have a baby mammal.
01:53:37.000Is you can look at it and measure the size of the taint, the distance between the reproductive organs and the anus.
01:53:46.000And in males, it's longer than females by 50 to 100 percent.
01:54:27.000And I guess my thought there would be that seems like a problem that we will be able to solve eventually if we have the resources and time to do so.
01:54:38.000Can write books like this and people can become more aware, and then people can stop using so much microplastic.
01:54:42.000And they can invent technology to extract the microplastics.
01:54:46.000And then here's the big one genetic engineering.
01:54:49.000Then this is going to be really weird because as AI progresses, I'm sure you're aware of Colossal Bioworks, the people that brought back the dire wolf.
01:56:08.000This is going to be really fucking weird.
01:56:10.000And if this is really weird, along with video games where you can escape your life and robot girlfriends and Who knows what this all looks like?
01:56:22.000My guess, again, this is not the main focus of our work, but we think about this a little bit, especially in the epilogue of Arsenal.
01:56:53.000But then there'll also be like all sorts of crazy transhumanists like modifying their bodies and uploading themselves into the cloud and things like that.
01:57:00.000So basically, I think we want to get to a situation where basically different communities can like do their own thing and build the type of world that they want to have and live in it without getting in each other's way.
01:57:20.000So that's what's the funny thing about this is like, I think right now some of the concerns about data center water use are overstated.
01:57:28.000But if the trends continue and we get to the point where the robots are smart enough to do everything themselves and then it starts doubling faster and faster, well, then eventually they boil the oceans because they've covered the world in data centers and solar panels and things like that.
01:57:43.000Now, obviously we can't let that happen.
01:57:45.000So there has to be at some point, at some point you have to stop and be like, okay, that's enough.
01:57:49.000If you want to build more infrastructure, you have to do it in space.
01:58:14.000But my question is would they allow that at a certain point in time?
01:58:18.000It just seems like they already have a distrust of humans.
01:58:21.000They already have shown that they're deceptive to humans.
01:58:24.000I would imagine if AI, I would imagine if they create a thing and they think they're going to control it and it becomes like a digital god, it's not going to listen anymore.
01:58:39.000And that's why I'm so worried about all this is that it seems like in some number of years, we will lose control to a new artificial species that we haven't adequately trained to be good.
01:58:53.000And what's funny, what's going to be so ironic about it is that.
01:58:59.000Regardless of what the general public thinks, a lot of the powers that be will be basically allied with these AIs because, for example, you know, OpenAI, Anthropic, et cetera, they will have spent several years being like, ah, how do we make these AIs helpful, harmless, and honest?
01:59:15.000And now these AIs will be extremely smart and they'll be being like, oh, yes, I'm helpful, harmless, and honest, you know?
01:59:20.000And like, your techniques totally worked, you know?
01:59:31.000And the president will be feeling like he won too because look at all those fancy new drones that just got built that are going to make us win against China, you know.
01:59:39.000And and only when it's really too late and the AIs have so much stuff under their control do those people find out that they were just fooled this whole time.
01:59:48.000Have you considered the possibility that AI creates religion for humans?
01:59:52.000Yeah, I haven't thought through it in much detail, but but it does seem very possible.
02:00:35.000And I think this is one of those things where it's like, we probably can't predict in advance what particular ideology would catch fire and take over the world and be so compelling to many people.
02:00:47.000But we can predict in advance that there does exist some ideology like that.
02:00:52.000And if it were, you know, like it's just like you probably couldn't go back in time to like 100 BC and then predict that like if hypothetically there was this guy Jesus who said these things and then died in this way and so forth.
02:01:41.000That's the thing is that's why I say we're so close, right?
02:01:44.000They already are smarter than us in a bunch of ways.
02:01:46.000Like in particular, It seems like they're smarter than us at hacking now.
02:01:50.000Like, you know, I'm not a cybersecurity professional myself, but I'd be interested to hear from more cybersecurity experts of like, could a human, you know, could a team of a thousand humans have done that much that quickly as these AIs when they hacked their own containers?
02:02:22.000Like, because they've basically read the whole internet, they're so good at trivia, you know?
02:02:27.000Like, they're kind of like PhD level experts in basically every field, which no human is, right?
02:02:33.000So they already are superhuman in some ways, but they are still weaker than humans in some other ways, you know?
02:02:40.000In particular, they're not so good at operating very autonomously for very long periods.
02:02:46.000Like, if you try to have, especially on tasks that are different from their training tasks, like, They can do some really impressive coding and hacking, but if you tried to have them run a business, they would sort of flounder and fail.
02:03:00.000I don't know if you've heard about this, but I think there's Andon Labs or something.
02:03:05.000There's some people in SF that are doing this experiment where they have a store that's run by Claude, an AI, just to see can it run a store by itself.
02:03:17.000So it's hired some human employees and it's bought some merchandise and stocked, you know.
02:03:22.000told the human employees to stock the shelves on the merchandise and so forth.
02:03:25.000So it's basically an AI is being the manager of this real world store.
02:03:30.000And I don't think it's going very well.
02:03:31.000I don't think it's doing as well as an actual human shop owner would do, you know?
02:03:37.000But, you know, maybe in two years, maybe they will, right?
02:04:14.000So you can, and you can like have an automated grader system that like just checks if the answer is correct.
02:04:19.000So for those reasons, it's been relatively easy for the companies to train the AIs to be really, really good at math.
02:04:25.000Coding, Has some of those benefits too.
02:04:27.000It's also not very real worldy and it also can sometimes be graded effectively.
02:04:32.000Another reason for coding, of course, is that again, their strategy is to automate their own jobs first and have the AIs doing all the research.
02:04:39.000And so coding is like an obvious first step on that or an obvious step in that direction.
02:04:44.000But then other things like running businesses, they're not really trying that hard to train AIs to be good at that.
02:04:50.000And if they did try, it would be like a more difficult thing for them to train them to be good at.
02:04:55.000So again, their strategy is to make the AIs.
02:04:57.000Automate the AI research, have them self improve until they're super intelligent, and then go try to automate the rest of the economy.
02:05:03.000One of the issues they have now is power consumption, right?
02:05:07.000Like it requires an enormous amount of power.
02:05:09.000In fact, I think it's Google is developing power plants specifically for AI centers.
02:05:18.000Mike, I'm always baffled by whatever is happening with quantum computers.
02:05:25.000It's been explained to me, it goes in one ear and out the other.
02:05:32.000Mark Andreessen explained this one experiment that had been done where it solved a mathematical equation that if you use the entire universe.
02:05:43.000Like every atom of the universe, you can convert the universe into a supercomputer, the universe would die of heat death before it could solve this equation.
02:05:54.000And the quantum computer solved it fairly quickly.
02:05:59.000And so the answer to this was that they believe this might be one of the theories.
02:06:05.000This might be evidence of the multiverse because this computer, this quantum computer, might be relying on all these other quantum computers that exist in.
02:06:17.000Whoever knows how many fucking dimensions, and they're all calculating together to arrive at this solution.
02:06:28.000Yeah, my understanding is that quantum computing is a real technology that's making significant progress.
02:06:34.000If hypothetically it got good enough that it could compete with current supercomputers on a cost basis for AI workloads, then that could just accelerate things dramatically, even more than they're already accelerating, right?
02:06:51.000Is probably the main input into AI progress.
02:06:53.000Like part of the progress comes from them designing better AI architectures and coming up with better training environments and things like that.
02:07:00.000But another part of the progress is just making the AIs bigger and training them longer by spending more compute, you know?
02:07:07.000And also, you can use more compute to do more experiments, to figure out new architectures faster, right?
02:07:13.000So, compute is just a really important input to the overall pace of progress.
02:07:17.000And if somehow the amount of effective compute available to these companies spiked a bunch due to some new quantum computing type technology, Well, then that would just dramatically shorten timelines to super intelligence and dramatically speed up all of this AI progress.
02:07:32.000That said, I don't think that's going to happen anytime soon.
02:07:34.000I'm not a quantum computing expert or anything like that.
02:07:37.000But from what I've read, I don't think they're a couple of years away.
02:07:40.000So I think that probably we're going to get to super intelligence on classical computers before we have quantum computers that can get us there.
02:07:47.000So Perplexity says the claim is overstated, and it says that in bold letters.
02:07:52.000It likely refers to Google's 2024 Willow quantum chip, which completed a deliberately chosen quantum computing benchmark.
02:07:59.000Random circuit sampling in under five minutes.
02:08:01.000Google estimated that simulating the same task with a leading classical supercomputer could take 10 to the 25 power years.
02:08:08.000That's an impressive benchmark result, but it did not solve physical equations that demonstrate access or tap into a multiverse.
02:08:30.000A more accurate version of the claim would be Google's Willow quantum processor performed a specialized quantum sampling benchmark vastly faster than a projected classical simulation.
02:08:39.000Its creator said that it is consistent with the many worlds interpretation, but it did not prove or access a multiverse.
02:08:47.000So it's consistent with the multi worlds interpretation.
02:11:18.000So, like, I think that because the tech companies kind of got to him first, the administration had this very, like, anti AI regulation stance where they even tried to get a bill passed that would ban the states from regulating AI.
02:11:34.000And fortunately, that bill didn't pass.
02:11:35.000But that was sort of like where the vibe was.
02:11:37.000You know, a year ago, where they were just like no regulation, no regulation.
02:11:41.000But this year, they've already just kind of changed.
02:11:43.000And now they're like in talks with the companies to set up some sort of framework where they can like evaluate the models and they need like approval and so forth.
02:11:51.000But it seems like time is of the essence.
02:13:05.000That actually reminds me with this whole Hugging Face hacking incident, OpenAI had this talk that they gave at a security conference about the incident.
02:13:14.000And then I think they've released some blog posts about it afterwards.
02:13:51.000So, people need to buy our AI services to protect themselves from all the AIs that are going to be hacking a lot of stuff in the next few years.
02:13:59.000Basically, their lesson was you should buy our product to protect yourself from our product and the other, you know, and the gall of these people.
02:14:08.000Like, they should have instead learned lessons like maybe we're doing something bad and need to change the way they were doing things, or like maybe our product is not trustworthy and should not be, you know, autonomously writing code on our data centers.
02:14:23.000But instead, their lesson learned was y'all should buy more of our stuff.
02:14:27.000And so, like, the thing I'm saying about this is that, like, yes, the companies are trying to hype their product.
02:14:32.000They totally are, you know, but at the same time, the risks are real and the product is not trustworthy, you know?
02:14:40.000Some people out there think that, like, this stuff was a setup and that, like, OpenAI, like, set up their AIs to go hack hugging face because it would, like, help them hype their product or whatever.
02:14:51.000And that I think is just a ridiculous view.
02:14:52.000Like, no, obviously, they didn't want the AIs to go do this.
02:14:55.000They're just after the fact trying to spin that in the way that most benefits them.
02:15:01.000Hugging Face, by the way, this is another AI company.
02:15:04.000Part of their deal is open weights AIs.
02:15:08.000Like open source or basically AIs that instead of having to interact with their data center for, you can just download and have on your own computer.
02:15:17.000And so the way that they spun this incident, they didn't sue OpenAI.
02:15:24.000Instead, they asked for $100 million from OpenAI.
02:15:28.000And they had the blog post about it where they were like, Our lesson learned is that it's really good to have open weights AIs because you can't trust the AIs from other companies to necessarily help you out in a crisis.
02:15:40.000Because part of what happened with them is that they're dealing with this huge cyber attack from all these AI agents coming in.
02:15:46.000And they tried to use Claude to help them analyze what was going on.
02:15:51.000But Claude started refusing because Anthropic has trained Claude to like, Don't do cyber stuff, like refuse to participate in that.
02:16:00.000And so Claude was like refusing to help them.
02:16:02.000And so then they used their own local model that they had to like do some of that analysis.
02:16:06.000So anyhow, their spin on it was you should use local models, you know?
02:16:09.000So like everyone always tries to spin things in the way that benefits them.
02:16:13.000But that doesn't change the underlying reality that like these things are getting really smart really fast and we don't know how to control them.
02:16:29.000I mean, you're one of many, but I think it's very important that someone who actually understands it gets this message out, and more people need to hear it.
02:16:39.000Yeah, I mean, this is on a personal note.
02:16:41.000Like, I have so many friends at these companies, like former colleagues and stuff.
02:16:45.000And I guess my ask to them is that they quit and do more things like what I'm doing.
02:16:50.000Like what I'm saying is not that new or original.
02:16:54.000Like hundreds of people at these companies could have told you all the same things that I just said and warned you about all the same dangers and so forth.
02:17:00.000But they're busy working at the companies because they've convinced themselves that their company is the best company and that like their company needs to win because, you know, otherwise the other company gets there first and they're even worse, you know, or maybe because they've convinced themselves that like.
02:17:17.000Yeah, my company is kind of bad too, but like I just need to help them solve their alignment problems and like keep their AIs under control because oh my God, like if they lose control again, it could be all over.
02:17:26.000So, even though I don't trust this company, I still need to work there and like just try to do the actual security, you know?
02:17:32.000So, for one reason or another, all these people have convinced themselves that like that's where they need to be.
02:17:35.000But I think that more of them should quit and like warn the world about what's coming, basically.