00:00:16.000So, since the last time we talked, we spent a lot of time where you were trying to explain to me simulation theory and why the probability of simulation theory is more likely than it not being a simulation.
00:00:32.000Yeah, it was what, five years ago or something?
00:00:42.000I mean, for example, back then, I think we did we even talk about AI?
00:00:47.000It probably came up a bit, but it's not like it wasn't this thing looming over civilization, which is really kind of fascinating when you think about the fact that it's only been six years.
00:00:59.000And in six years, like, what a massive jump in some new technology in our life.
00:01:06.000Sort of like the internet, where it crept up on us.
00:01:09.000We just accepted that it's a thing, but that this thing has gotten massively entangled in every aspect of society and every aspect of people's lives in a very short period of time.
00:01:23.000Yeah, I mean, things are like so much is happening now that it's kind of a full time job just to monitor the situation.
00:01:43.000I mean, I think I take both cases seriously the sort of the risk side and the big unlock if we get things right.
00:01:54.000I feel like we have the potential, like we're on a whitewater raft.
00:01:59.000We have the potential to get to our destination, but we also have the potential to flip over and try to figure out how to get to shore and freeze in cold water and sort of rebuild.
00:02:45.000It's just so strange how quickly it snuck up on us, and that there's two narratives that we hear.
00:02:52.000The one narrative is we're in real trouble, and that this thing is going to take over every aspect of society, and it's essentially going to be a superior life force, a superior intellect that exists amongst us that we created, and we don't think that's wise.
00:03:07.000And then there's the other side that is saying things like what Elon says, where he's saying we're going to have universal high income.
00:03:15.000It's going to be so much prosperity that no one's ever going to have to toil again.0.93
00:03:19.000There'll be no more third world countries.
00:03:23.000We can eradicate poverty with the resources that we have on earth, and we can change what it means to have to work just to provide yourself with food and housing.
00:03:51.000Mark Andreessen, all these people that have this rosy view of it, they're all invested heavily into it.
00:03:59.000So, when someone like you, who's not necessarily in that camp, that is more of a true objective analyst of what's going on, when you have a positive aspect or a positive viewpoint of it, I get a little more excited.
00:04:14.000Yeah, well, I mean, the truth is we don't know how it will pan out.
00:04:19.000So, I mean, I think there are these scenarios where we unlock this enormous boost, both to economic productivity, but then across medicine, entertainment, environment, travel, all kinds of things.
00:04:35.000And like a tsunami of wealth just kind of flows through and lifts all boats.
00:04:41.000And then the idea of human work becomes an anachronism, and where you have machines that can do everything that we can do physically and mentally and do it much better and cheaper.
00:04:54.000And so I think in those scenarios where this really works, the transformation is a lot deeper.
00:05:00.000I mean, so that kind of layers to the onion.
00:05:03.000So, like, the most superficial level is, well, they automate their jobs.
00:05:06.000So, what are people going to do for workers?
00:05:09.000What are they going to do for workers?
00:05:11.000First, like, the most superficial level is people just wonder, where will I get a job if the robot replaces me?
00:05:17.000And then the superficial level of that conversation, I think, is, well, you need to retrain workers so they can work in other fields.
00:05:23.000And maybe there needs to be, I don't know, employment insurance whilst they are being retrained or something like that.
00:05:29.000But once you think through where this ultimately leads, you think it's not just a few jobs, I think, but it's really everything that humans can do to a good approximation.
00:05:40.000With maybe the exception being where the consumer has a direct preference that a particular product or service be done by a human, like priest, prostitute, and politician.0.99
00:06:11.000And so then you get to these more profound questions, I think, about meaning and purpose and what does a human life look like at technological maturity.
00:06:43.000Like, do you have to do everything in the material world, or can you find happiness in a virtual world that doesn't currently exist, but we could clearly see the technology if it expands?
00:07:02.000The possibility of experiencing a matrix type reality.
00:07:07.000Yeah, I think I see where you're going.
00:07:10.000But yeah, and I mean, I think like different strokes for different folks.
00:07:13.000If you imagine a world where presumably these virtual worlds will be very rich and deep and fascinating, but some people might just like the idea of, you know, climbing the real Mount Everest rather than, you know, being in it.
00:07:45.000There's nothing to do because AI is taking care of all aspects of society.
00:07:48.000Other than recreation, there's nothing to do.
00:07:51.000And then this one recreation comes along that's not reality, but it's way more fulfilling and exciting than any other aspect of reality.
00:07:59.000And, you know, an example of that would just be in a very minor way your phone provides you with that sort of an escape, and it's not even that thrilling, and yet it's massively addicting.
00:08:12.000Your phone, people are on their phone six hours a day.
00:08:15.000If we come up with A virtual reality that's way more exciting than regular reality, everyone's going to hop on in.
00:09:02.000Or in video games, like you could have like a really intense, every straining, every fiber to try to succeed in this virtual environment.
00:09:08.000Or like one where you're just kind of floating on some cloud in a drug like state.
00:09:13.000But I don't think people are going to be interested in that because just the way the human mind works, like what kind of video games are people attracted to?
00:09:20.000It's because a video game is essentially a proximity.
00:10:11.000You're not really doing much and it's more kind of zoning out.
00:10:14.000But most people aren't interested in those.
00:10:16.000Most people are interested in Call of Duty, they're interested in these wild first person shooters where you're running down hallways and everything's exciting and thrilling.
00:10:37.000But either way, I think a lot of the choices that people would make currently depend on what they are sort of designed to respond to by feeling good about it.
00:10:47.000So, like, if you enjoy one thing, you choose that.
00:10:51.000In this condition of technological maturity, if we imagine sort of a future civilization that has developed all possible technologies to their maximum.
00:11:03.000Then, amongst the affordances, amongst the things they could do is like they wouldn't just have control over the external environment, things around them, but also their own biology and brains.
00:11:15.000So, if you were one of these people, and like maybe right now the only way you get enjoyment is by, you know, getting like crazy drunk or doing sort of things you don't really approve of, maybe, but that's actually what sort of lights your fire.
00:11:31.000Imagine redesigning yourself to take the same kind of pleasure, but in some kind of contemplating the beauty of the universe or like appreciating the goodness in the heart of others or like some more sort of noble aspiration, like solving abstract mathematics rather than, you know, playing first person shooters.
00:11:50.000You would have a choice, like whether you would get your thrills from one kind of activity or the other.
00:11:57.000Yeah, which brings us to the question like, what does it mean to be human?
00:12:02.000Beauty and joy in interesting and fascinating things, and how to be a better person, and all the different aspects of human life that we think about when we think about people and we think about noble aspects of humanity.
00:12:19.000But doesn't that have to exist in conjunction with the worst aspects of society for us to appreciate it?
00:12:27.000It seems like this is a part of the human condition we have to have crime so that we can appreciate peace.
00:12:42.000They're always both together in constant conflict, and we're always nobly hoping that the good wins out over the bad.
00:12:50.000This is part of the struggle of being a human being.
00:12:53.000If we just completely eliminate that struggle, we're going to have to be a different thing because what we are is this very strange territorial primate who's Endlessly curious.
00:13:09.000This territorial primate is moving in a certain direction, and that direction seems to be a better society, seems to be over time.
00:13:19.000If you look at statistics from thousands of years ago to today, there's less violence, there's better medicine, there's better education.
00:13:26.000It's moving in a better and better and better direction.
00:13:38.000Because we're going to be a different thing than what we are right now.
00:13:42.000And if you love music and if you love art and if you love novels and all these different things that come out of the human condition, well, those things come out of struggle.
00:13:53.000Those things come out of confusion and pain and heartbreak and love and joy and all that stuff all piled up together.
00:14:24.000There is a kind of paradox embedded in our efforts to make progress.
00:14:31.000So, like, there are all these kind of scientists and people working to develop better technologies, and throughout the economy, you know, in some company, maybe you try to figure out how to make some process a little bit more efficient so you can serve customers better, and all of this is.
00:14:46.000Like, if you sort of extrapolate that to its logical endpoint, right, you would imagine we would have perfect technology that can do everything, solve all the problems, like presumably with AI and automation.
00:15:00.000But then you end up in this condition where there is kind of nothing left for us to do, you might think.
00:15:06.000And so, although it looks like we have these strong reasons to push forward in this direction, if you actually look at the endpoint, if we succeeded, To many people, it will look kind of unpalatable and like this kind of future where, you know, all the problems are solved.
00:15:21.000And nevertheless, that does seem to be, you know, the direction that we are headed in, probably the direction that we should be headed in.
00:15:30.000Wouldn't that be a better goal if there was zero murder, zero violence, zero crime, zero lying, zero corruption, and human beings all worked in coordination with each other, just like in a sense of unique sinking and harmony?
00:16:04.000It's just, it looks bland if you think about what we're experiencing now like all the excitement and the weirdness of uncertainty and of not knowing, you know, and also the negative aspects of our life in conjunction with the positive aspects of our life.0.99
00:16:19.000So we contrast the two of them and you really appreciate good people after you're around a bunch of fucking assholes.0.98
00:16:25.000You know, if that doesn't, but it doesn't have to be that way.1.00
00:16:28.000This is just what we are dealing with.
00:17:47.000They all think like we have to make the most money because we have shareholders and we have a responsibility, and these people need to make more money every quarter.
00:17:53.000Yeah, although sometimes you do this, you compete for talent.
00:19:59.000I mean, so it used to be the traditional idea, like a British aristocracy, like that you wouldn't have to work for a living.
00:20:05.000It was kind of an unfortunate necessity for many people that they had to work for their daily bread, right?
00:20:11.000But the ideal way of being human was that you just had money and then you spent your time doing other things.
00:20:17.000Like, you know, maybe you were involved in politics or maybe you had like an art collection or you did your gardening or entertained friends.
00:21:02.000Did they explain the value of hard work that it's actually good for you?
00:21:06.000And that if you just find something you really want to do and you concentrate on it and really work hard at it, it's actually very fulfilling.
00:21:13.000Yeah, so you could imagine, for example, the education system in this world where we no longer need to work.
00:21:18.000Like, presumably it would need to be redesigned from scratch, basically, because right now it's a kind of machine, right?
00:21:26.000So you take children coming in kind of on a conveyor belt and then some processing is done.
00:21:38.000And then they are meant to be productive workers that you could put in an office and they will.
00:21:43.000So, this kind of is an unfortunate fact about the current condition that I mean, the world needs a lot of these office workers that can do all these tasks in the economy.
00:21:52.000So, like, it's we have to have until it doesn't.
00:22:10.000Like, imagine cultivating the art of conversation, appreciation for art and music, hobbies, physical wellness, like nature, the ability to set your own goals and take your own initiatives to develop true friendships with people.
00:22:30.000These are things that are not taught in school, but you could imagine that being the curriculum, like spirituality, like all kinds of things that would then equip people to sort of.
00:22:40.000Use their freedom, their wealth, their free time for some kind of actually meaningful and beautiful activities.
00:26:04.000It's like, who's the best at this thing?
00:26:06.000Like a bunch of friends who play golf, like Jamie plays golf all the time.
00:26:10.000Golfers are all like comparing each other's scores and they're all, they're playing, they're competing in this game.
00:26:16.000And that, they think about that more than they think about work.0.99
00:26:19.000Like people who love golf, they fucking hate work.0.98
00:26:21.000Like I used to say that about comedy back when I first started.0.98
00:26:25.000One of the things that I noticed is the guys who really got into golf, their careers kind of stalled because they were more excited about playing golf than they were about writing jokes and going on the road.
00:26:37.000So if the average person doesn't need food and housing anymore from labor, if that's gone and now you just get it, and so now you could just go do things, we just have to teach people to be excited about stuff.
00:26:50.000We have to teach people the value of curiosity and finding things that are interesting to you, and then the value of just education for the sake of learning things because it's interesting.
00:27:02.000Just pure satisfaction of curiosity, which is a beautiful thing.
00:27:06.000Like that would be great for everybody.
00:27:08.000If we instead of Learning things because your teacher tells you you have to learn it.
00:27:13.000Well, there's always going to be people that just naturally gravitate towards mathematics and they're really fascinated by mathematics.
00:27:19.000And there's always going to be people that are just naturally gravitating towards history.
00:28:16.000I think there's probably also some differences in how different folks are wired.
00:28:20.000Some people get bored very easily, others find everything fascinating.
00:28:23.000Some people are just naturally depressed or have low mood, others are sort of evolving.
00:28:28.000They were coached at a very early age the value of exercise.
00:28:35.000And if they started running, if they started doing yoga, and they started feeling much better, and they weren't naturally depressed anymore.
00:28:41.000And then you change their diet, and you start adding in vitamins and nutrients, and stop giving them processed foods, and all of a sudden they feel better.
00:28:48.000Because the concept, even of depression, like what does that mean?
00:28:55.000And when you're talking about people that aren't taking care of their bodies, and that's not thought of as a primary factor.
00:29:01.000In why depression exists in the first place.
00:29:04.000Well, this is a very unscientific approach, then, because we know that it has a profound impact on human beings.
00:29:10.000So we're pretending that there's like this one size fit all.
00:29:13.000No, I think that should be also a part of teaching people how to be a human being.
00:29:18.000Chase your curiosities, but also, this is why you're not feeling well.
00:29:22.000And that if you just develop this ability to get some momentum going and just show up at that yoga class every day, you'll feel way better and you don't have to take a pill.
00:30:15.000We're not asking people to breathe underwater, right?
00:30:17.000We're saying that there are people on earth that live like that.
00:30:23.000And this idea that living just for a paycheck so that you can cover your food and your housing, which we've always thought of as being just an undeniable part of being a person, an adult.
00:30:38.000You have to pay for your food, pay for your housing.
00:30:39.000But if we do get to a point where the structure of society is now run by hyper intelligent artificial intelligence, you wouldn't need most jobs that people have to do that suck.
00:30:53.000In order to get our society to this point, if you wanted an iPhone, you needed some people that were in a factory somewhere putting it all together.
00:31:03.000You needed someone who's designing it, you needed someone who's sitting there in the office trying to figure out how to market it, you needed all those jobs.
00:31:11.000But when we don't need all those jobs anymore, then things are going to be very interesting.
00:31:14.000And that's what I'm saying we need to become a different thing.
00:31:51.000Currently ill suited for really thriving in an environment of abundance and for enjoying life.
00:31:59.000Because, I mean, both at deep biological time scales and during the lifespan of a single individual, there are all these pressures, necessities that kind of force us to become a certain type.
00:32:11.000So, I mean, we talked about the education system, training people to be the kinds who can sit at the desk all day long and perform tasks.
00:32:59.000And it's kind of been necessary because there needed to be this motivation to keep striving for the next thing.
00:33:07.000Now, once you've actually achieved all the things, though, then maybe that becomes kind of dysfunctional, right?
00:33:13.000Like, why keep wanting to strive for the next thing if all the things have already been achieved, at least in a certain domain?
00:33:20.000And so I think as we move deeper into this hypothesized future, like, Where we get all these magical technologies, then at some point, probably some transformation of human nature would have to go along with that.
00:33:36.000First, maybe cultural changes to sort of equip people for a life of leisure.
00:34:00.000We don't know because we haven't lived for 200 years, but that could easily turn out to be the case with a human.
00:34:04.000So, then maybe at that point, you would need to sort of do something to sort of add more cognitive resources, more flexibility, some sort of psychedelic reset or something to sort of keep the flexibility going for longer periods of time.
00:34:20.000I'm glad you said a psychedelic reset because it would be very funny if the main tools that we have for navigating this are all illegal.
00:34:30.000And specifically with psilocybin and DMT and probably Ibogaine as well.
00:34:36.000Like, if we wanted chemical tools to navigate a new reality, those are probably the best ones that we have available, and they're all illegal.
00:34:47.000I think if we're going to be able to navigate this correctly, we have to kind of change the way we interact with each other, what it means to be a person.
00:34:56.000But I wonder how much of the conflict that we have is a direct result of this inherent struggle that so many people have.
00:35:04.000I mean, there's a direct correlation between extreme poverty and extreme crime, you know, specifically in this country.
00:35:10.000If you look at the areas of extreme poverty in this country, they're also the areas with extreme crime.
00:35:17.000I wonder how much of that would be completely alleviated with a complete lack of poverty.
00:35:22.000We've always assumed that if you're going to have a functioning society, you're going to have slums.
00:35:37.000Well, it's because some people are always going to be making bad decisions and some people are always going to be going down the wrong road and crime and this and that.
00:35:56.000It's again, not asking people to breathe underwater.
00:35:58.000We're just trying to figure out why some people never engage in crime?
00:36:03.000Why do some people live these really fulfilling and interesting lives?
00:36:07.000Probably because they were exposed to it when they were really young.
00:36:10.000Probably because they weren't exposed to very bad environments and very bad crime and very bad poverty.
00:36:16.000And how much of that would change if there was no more poverty?
00:36:20.000It sounds like such a little fairy tale, childlike, oh, one day we'll have no poverty, but that's doable.
00:36:28.000If everyone is alive, right, okay, that's alive right now is not starving to death, that means we figured out a way to at least, the very least, get you resources so you could feed yourself, right?
00:36:40.000No matter how dysfunctional things are, all that has to do is you get ramped up a few steps.
00:36:54.000But it seems like there's a lot of people that find purpose without having a financial price tag attached to it, just by what we're talking about with golf.
00:37:03.000Or you could be really into writing books.
00:37:07.000You could write books all day long, and people are always going to want human created fiction.
00:37:12.000People are always going to be interested in the way other people think about things.
00:37:15.000You'll find there's plenty of things to do, there's plenty of games to play, and plenty of skills to learn.
00:37:22.000The idea that the only reason why we work is to eat and to not get rained on seems nuts.
00:37:30.000Yeah, it is, I guess, to some extent an open question to what extent people will always want to read human fiction or to prefer the human generated outputs.
00:37:43.000It might be just because current AI generated writing is kind of lacking in various ways.
00:37:52.000But if it became really good, then maybe it would just be much more fun to.
00:38:00.000Like, if a movie made by an AI might just be so much better and richer, and the lighting is perfect, and the dialogue is sharp, and it's more funny and deep.
00:38:09.000Then you go and watch this human produced thing, and it's going like most people don't go and watch sort of film school students' productions.
00:38:18.000Right, but then there's that movie obsession.
00:38:20.000Wasn't that movie made very inexpensively?
00:38:26.000And that's part of the thrill of it, is that this guy who was like a YouTuber, right, who created this film.
00:38:32.000And now this movie's a giant hit, and everybody's like super excited about it.
00:38:36.000Now, if that same movie was made by AI, I wonder if it would have that impact because it doesn't have the human connection.
00:38:43.000So, that's possible that we keep getting interested in these things because we sort of are really entangled with the human world much more than we are with the world of AIs.
00:38:56.000And I mean, for the same reason, you might be more interested in if like your brother or friend did something.0.99
00:39:02.000You might think, oh, this is interesting.
00:39:15.000When your nephew gets on first base, like, way to go, Bobby!
00:39:19.000So I think these kind of social entanglements that we have will be a big part of what gives structure to lives in this condition where the external constraints are removed.
00:39:33.000I think that there is a great value for the human mind, for whatever reason, in getting better at things and learning things.
00:39:41.000And I think that if we could instill that in people at a young age, I think.
00:39:46.000It would be fairly easy to get people to pursue that path.
00:39:50.000But we have to completely revamp our education system, which should have been done a long time ago anyway.
00:39:56.000The idea of having unenthusiastic, underpaid people being any percentage of what children encounter for most of the day when they're in their most formative period is insane.
00:40:08.000It should be a rich, exciting, enthusiastic journey in how to be a better person and how it's exciting to learn things.
00:40:17.000And I was excited to get better at things and about how anybody can get better at things.
00:40:30.000I mean, I only discovered, like, I also hated books and everything like that for my first 15 years of life because I associated that with school.
00:40:39.000So it's only then randomly one day I happened to walk into the local library for no reason and pulled out a book here and there and then.
00:40:49.000I discovered that there was this whole world of science and idea and literature and all of that, like very, very different from what we were doing in school.
00:40:55.000And then that kind of opened the gates for me to this.
00:40:58.000Imagine what a head start you would have had if you had like a different kind of education with super enthusiastic people who really love teaching children and are really good at it and that we reward them and that it becomes a very prestigious position to be in rather than what it is now.0.98
00:41:16.000If you talk to some guy and he goes, Oh, I'm a high school teacher, you're like, Oh, poor bastard.0.90
00:41:23.000Like, good for you that you're doing that, but also I guarantee you could probably be making a lot more money and be happier doing another job.
00:41:31.000And so that's a terrible way to start life off.
00:41:34.000And if we just revamp that, then you have a bunch of people that are interacting with life in a very different way.
00:41:40.000And instead of being thought, I have to get a job someday, I have to get a job after school anyway to help my mom pay the bills because we're struggling.
00:41:49.000So I got to contribute to the household even though I'm 16 and I can't hang out with friends.
00:42:35.000I think that's kind of inspiring, but I think the train doesn't stop there.
00:42:41.000So if we think even more further into this kind of condition of technological maturity, I think in addition to sort of freeing up people, you know, like making it easy to produce the food and the houses and like cars that go.
00:42:56.000Faster and don't pollute and all of this stuff.
00:42:59.000Like, if you think through what maybe a technologically mature civilization could do, like, so a lot of these, like, learning, for example, is something that gives people purpose, but maybe you would have the ability to sort of download knowledge at the click of a button.
00:43:15.000So rather than, like, if you want to learn advanced mathematics, now you have to study for years, right?
00:43:20.000Do like books with exercises and apply yourself, and then eventually you sort of unlock.
00:43:36.000Yeah, so if you had, like, you know, maybe some kind of nanobots that could infiltrate your brain and then change the synapses in just such a way that you're not the same as you were before, except you have these concepts from abstract mathematics.
00:43:51.000You know, after 20 minutes or something like that, some superintelligence works out how to change your synapses to this new condition.
00:43:59.000And if you were inclined to do various things because they give you joy and pleasure, like, you could also think, well, I could do those things and get the joy and pleasure, or I could just push this button that activates the same joy and pleasure.
00:44:13.000So there would be these shortcuts to everything.
00:44:16.000And it looks like you would have a post instrumental condition where there would, at least at first sight, seem to be no reason to do anything for the sake of achieving something else because there would always be this shorter path to that other thing, pressing the button.
00:44:33.000And so that's a kind of deeper form of redundancy.
00:44:35.000It seems that all forms of human instrumental effort.
00:44:41.000You could still go to the gym every day and sweat for an hour, or you could just take the pill that induces exactly the same physiological effects and the mental effects, like the calm or whatever you feel after.
00:44:58.000Or maybe even better, genetically engineer something.
00:45:05.000And so then I think you're getting more into these fundamental questions of value, philosophical questions, but what ultimately is it that.
00:45:14.000And what makes a life good for a primate, because that's what we are, or a primitive version of primates?
00:45:21.000I think we're going to become something different.
00:45:23.000If we do it through technologically induced evolution or biologically induced evolution, we're still moving in a general direction.0.95
00:45:31.000Just biologically, if you look at ancient hominids and look at us, we're clearly very different.0.97
00:45:36.000So we're moving in a general direction anyway.1.00
00:45:39.000When you look at the grays, like these prototypical aliens that everybody sees, the big heads and the skinny bodies, what do they look like?1.00
00:45:46.000Well, they look like if we keep going, that's what we're going to look like.1.00
00:45:50.000We're going to be these genderless, sexless, muscleless little skinny things with giant heads that communicate telepathically, which is probably where we're moving.
00:45:58.000And if we're moving that way, maybe we think of them as being an alien from another.
00:46:03.000Maybe that's just the general direction that primates go once they figure out technology.
00:46:09.000They eventually realize well, first of all, the idea of having to have a male and a female is kind of crazy, right?
00:46:19.000Well, what if we don't reproduce like that anymore?
00:46:22.000What if all reproduction is done through engineering and there's no more sex, so there's no more lust, so there's no more jealousy, so there's no more ego, there's no more anything.
00:46:33.000And then there's a hive mind because technology has advanced to the point where the reason why they have these big giant heads, they're essentially locked into everybody all around them all the time.
00:46:41.000Yeah, well, is that what we would want though?
00:46:45.000Not us, but I don't think we'll be us anymore.0.99
00:46:48.000I think this idea like if you went to chimps.
00:46:50.000And you said, hey, dude, one day you're going to be on a plane eating peanuts, flying over the ocean.1.00
00:47:34.000Now, the question is the thing that we become or that becomes, is that a result of us choosing how we want to be, or is it just these kind of impersonal forces like evolution, selecting certain types that might end up leading to outcomes that we actually don't want?
00:47:56.000And so, I guess the hope is that we would be able to develop along a path that preserves and develops.
00:48:04.000The things that we actually value about being human, maybe amplifies them, then maybe adds other things.
00:48:11.000So there are many different possible things you could develop in the future, but that we sort of select those that actually make us better rather than just randomly different, that we can sort of grow into our ideals.
00:48:24.000But it's so funny how much value we put in human condition, how much value we put in meaning, because it is valuable to us.
00:48:34.000When you think about what that means in terms of the amount of energy it produces, the amount of impact that it has versus the structure of the universe itself versus black holes and stellar nurseries and things that are just infinitely more spectacular than the human condition.
00:48:52.000But to us, it's like, oh, what is meaning?
00:48:57.000Like, well, what is meaning to you is you're a finite biological organism that has to find meaning because you're sort of.0.88
00:49:05.000Tromping around through this weird world where eventually you're going to die and you're going to leave your mark, and maybe reincarnation is real, maybe the afterlife is real, and nobody fucking knows.
00:49:23.000I mean, we think of us as like first you become stronger and more capable for 15, 20 years, right?
00:49:32.000Then biologically you stagnate, and then Maybe you keep accumulating sort of knowledge and skills for another few decades, but just as you have started to acquire a modicum of wisdom, your brain starts to rot.
00:49:45.000And then everything is erased either by Alzheimer's or death.
00:50:05.000The human lifespan, so you could continue to grow up, not just kind of reach 20 and then plateau there for the rest of your life.
00:50:12.000But what if we could continue to develop so that, like, at 80, you were as much stronger and, like, be able to understand more and move better and, like, just have the same capacity increase as you had, like, between zero and 20 and you just kept going?
00:50:28.000Well, these life extension scientists that are working on these things, guys like David Sinclair, like, they believe that that's not just a possibility, but an inevitability.
00:50:37.000Yeah, well, so that's more like, I guess, preserving, preventing or delaying that decay.
00:51:12.000And if they can figure out how to make people, well, there's already genetic engineering that's being done in terms of increasing potential intelligence and IQ.
00:51:22.000I'm sure you know about that thing that happened in China where this one doctor got in trouble.
00:51:30.000Because he had genetically engineered some babies to be inoculated to HIV, but it also, at the same time, gave them a far increased potential IQ.
00:51:44.000It remains to be seen whether or not it actually works as these guys become like 20 and 30 and we start putting them in chess tournaments and see if it really did make them smarter.
00:51:54.000But if the possibility of that is already being studied, they already understand the differences between what.
00:52:04.000What's possible, what we understand, and that increases every day.
00:52:08.000They understand more every day, more is possible every day.
00:52:12.000If that just keeps going, well, you have a different version of what the human consciousness is.
00:52:16.000You have a different version of the human mind.
00:52:27.000And then once you do have superintelligence in machine substrate, then that can unlock all kinds of technologies.
00:52:35.000Including these biological technologies or nanotechnology or a host of other things that could then sort of bring us up if that's how we decide to go.
00:52:46.000And uplift us into different, like either biologically enhancing us or like uploading us into.
00:52:52.000But again, it's that white water river raft.
00:52:55.000It's like we're going down this white water and we might make it out of this or it might crash.
00:53:01.000You know, one of the weird things about, I'm very fascinated with, Ancient civilizations.
00:53:09.000One of the weirdest things about ancient civilizations is when you go really, really far back, a bunch of them have these depictions of kings that ruled for thousands of years.
00:53:20.000It's very strange stuff because the Egyptians and ancient Sumerians, there's a bunch of different depictions of these kings that lived lives that were way longer than biological humans lived.
00:53:39.000Or the flood story, and then after the flood, biological life decreases pretty radically, and it seems to get back to like a normal version of what we assume now, which is like 100 years.
00:53:50.000And one of the weird questions that these alternative ancient historians have is are we missing the possibility that there was a hyper advanced civilization that existed tens of thousands of years ago, and that what we're seeing now is not a linear progression from caveman to human being with an iPhone in 2026.
00:54:12.000That along the way, there might have been a very high level of sophistication.
00:54:18.000The evidence of that might be the pyramids and some of these other ancient structures that are mind boggling.
00:54:23.000As much as you try to explain them away, when you're dealing with 2,300,000 stones, some of them are cut from a quarry 500 miles away, 80 ton stones that are in the ceiling, perfectly cut granite where it looks like they have diamond drill holes in them, it seems like there's some lost technology.
00:54:42.000Every society has this flood myth, and every society that has this flood myth.
00:54:47.000Especially these ones that were very advanced somehow or another, like ancient Egypt, they have these depictions of people that lived way longer than people live today.
00:54:58.000And I wonder if human beings one day will realize, like, oh, if you keep going long enough, 100 years is silly.
00:55:06.000Like, people can, they'll just figure out what it is that makes people age and die, fix that, and you'll live an insanely long time.
00:55:14.000And then if people live an insanely long time and their capacity for reason and logic increases, Their capacity for intelligence increases.
00:55:21.000Then you have these insane technologies that were required to do things like build the pyramids.
00:56:03.000Like, are we really like reached the level of, like, if you want to run an airplane, you have to go through these certifications and stuff, right?
00:56:25.000But yeah, having the ability to kind of try different things and maybe do like have a kind of opportunity to do a do over.
00:56:38.000If the life turned out to be like every 50 years or something, you have a chance to kind of do a do over, but it's the same life.
00:56:47.000But you can do whatever you want, and you don't have to be constrained to this idea that you only have a certain amount of time and you want to retire by your 65.
00:56:55.000We might look at that as being like one of the first completely archaic notions and the reason why people got it all wrong.
00:57:01.000Like, well, society will be better because people are going to live way longer.
00:57:05.000And if you think about how much smarter, how old are you?
00:57:15.000But if you think about how much smarter you are at 53 than you were at 13, you know, now imagine how much smarter you'll be at 353 or 1,000 and 53.
00:57:26.000And if that actually becomes possible, If a person can live to be 100 years, why can't they live to be 4,000 years?
00:57:33.000If they can figure out what makes people age, if we really can genetically engineer human beings, that's not, again, that's not breathing underwater.
00:57:43.000And if you extend life and extend intellectual capacity and your ability to learn and grow, holy shit, you're dealing with a 2,000 year old person.0.95
00:57:52.000That's a completely different kind of thing.0.91
00:58:43.000So, I think a lot of this is Stockholm Syndrome, that you are kind of faced with the inevitability of aging and decay and have been for thousands of years.
00:58:53.000So, you develop a kind of romantic scaffold that reconciles you to the inevitable.
00:59:17.000We don't yet have the therapies, but there are certainly like investments and research and stuff we could do that would hasten the arrival.
00:59:24.000And I think, in fact, we should have started on a big sort of, you know, Manhattan product to defeat aging like decades ago.
00:59:33.000And maybe we would have better therapies by now then.
00:59:36.000But for a long time, aging was not seen as a problem, only the specific diseases that are the result of the aging.
00:59:44.000Like, so you have like huge research products for Alzheimer's and for heart disease and for cancer.
00:59:49.000But if you look at the main common cause of most human disease, it's senescence.
00:59:54.000Like, you're much more likely to get any of these when you're 80 than when you're 20 because your cells gradually decay and gunk builds up.
01:00:09.000So, rather than just trying to put out the fires one by one after they occur, like, at some point you just need to sort of try to prevent, like, maybe you need a bit of rain to sort of.
01:00:20.000It's kind of amazing how much we've accomplished as a society, as civilization, when you think about the fact that people only live to be 100.
01:00:32.000Got rockets and satellites and cell phones, and people only live.
01:00:36.000So it's like everybody has to build on everybody else's intelligence and everybody else's understanding of the world and develop new things, and everybody has to learn those new things.
01:00:45.000And each brain is so small that it can only learn a small little specialty, right?
01:00:50.000Although it's not entirely clear with.
01:00:53.000With humans, if we had been much more long lived, like the way we currently are configured, maybe we would just have gotten too stuck in our ways and sort of stagnated into some sort of ultra conservative society where nobody's allowed to do anything different.0.86
01:01:08.000Or the old geezers that have their way of doing things just remain forever in charge.0.96
01:01:28.000It's a long path from running around in the forest to these.
01:01:33.000Look at these advanced chip technologies and the whole global supply chain, where thousands of people are working to develop just one little tool that then feeds into the ability to make another tool that eventually makes these leading edge AI chips.
01:01:49.000Where they're layering things four atoms deep.
01:02:24.000At the same time, all over the world, blowing up people.1.00
01:02:29.000Yeah, but maybe a lot of that frantic, stupid, Illogical behavior is because we're so finite.0.98
01:02:38.000Like we're in a rush that, you know, we realize we only have a certain amount of time to get things done.0.96
01:02:43.000And so, this sort of accentuates the desire to control resources and to cement your immortality and to do these things that people love to do put their name on buildings.
01:02:54.000There's a thing that people like to do that's almost like cementing their immortality.
01:02:59.000Maybe there'll be less of a desire to do that if people lived way longer.0.99
01:03:03.000And then you would have to assume if you can engineer humans to live longer, you could probably engineer a lot of stupidity out of us.0.98
01:03:12.000They could just find out why people behave like, what if you could just eliminate lying?0.99
01:03:17.000What if there's like a genetic solution to lying?
01:03:23.000Well, I guess they're a really good lie detector.
01:03:26.000Yeah, but I mean, what if you can genetically engineer out the desire to lie?
01:03:32.000Well, I mean, if you had a perfect lie detector, there would be no point in lying because people would just see it immediately, right?
01:03:40.000Or if you could read each other's minds, it'd be fruitless.
01:03:59.000Well, one of the weirder things that Bob Lazar said again, I don't know if you're into this UFO stuff, but he was one of the guys that was a whistleblower that said he back engineered crafts for the United States government in the 1980s.
01:04:11.000They don't have any controls in these crafts, supposedly, and that they're all powered by the minds.
01:04:17.000Of the beings that are running it, and that they have it's almost like these crafts are alive and they have some sort of a syncing with the thing.
01:04:24.000That instead of like pressing buttons and working a joystick, they just sync with this creation and then they can propel it with their minds.
01:04:34.000Yeah, the minds become the computer that moves this thing around.
01:04:39.000I'm a bit skeptical, I haven't looked into that.
01:04:55.000So I think that the kind of possibility of engineering something like that with sufficiently advanced technology would be there, like all kinds of stuff.
01:05:31.000Yeah, but even with our current understanding of what physics permits, that's still an enormous space of designs and types of life and being that you could imagine instantiating.
01:05:46.000You could have, you know, like a Dyson sphere.
01:06:47.000And then after, does he remember any of that?
01:06:51.000Like, was he even really aware of any of this while he was driving?
01:06:55.000It's like a little diffuse sense of body and some murky perceptions floating through, maybe some confused abstract idea that we don't really understand.
01:07:05.000That's kind of the consciousness that can fit into this coconut sized biological organ that we think.
01:07:10.000And we think, wow, we are so conscious.
01:07:12.000But Like, imagine this Dyson sphere consciousness, or like a mind that maybe spans a galaxy.
01:07:19.000I think the difference between the sort of awareness that it could have and our awareness might be like bigger than the difference between our awareness and like whatever a flea has or something like that, way bigger.
01:07:32.000So it could be this like transition where we develop super intelligent minds that for the first time is really life waking up and becoming truly aware.
01:07:44.000And that we are a little bit sort of overpride, proud in our own specialness when we think that we have achieved something close to the maximum level of that we are the standard by which consciousness should be measured.
01:07:59.000And we are this kind of feeble, confused, murky glimmer that is barely sentient at all.
01:08:08.000So that's like, I think, maybe the big challenge for our era, like giving birth.
01:08:15.000To superintelligence and then hopefully shaping and nurturing it and steering it so that it becomes a positive thing, both for us ourselves and also for it itself, and also for whatever other, if there are other super beings somewhere in the world or in the cosmos, that it sort of is able to get along with us and contribute positively at the cosmic scale.
01:08:42.000And that's a very multifaceted challenge, but I think that's kind of seems to be what's going on.
01:08:50.000It's hard for people to think that far ahead.
01:08:53.000You just think of intelligence being something superior to what we currently experience.
01:08:57.000But when you're talking about a computer or a being, a conscious being, that is infinitely more powerful than anything we can imagine, that seems to be what.
01:09:13.000If everything keeps going in the same general direction and AI increases its power and we figure out new ways to power it.
01:09:21.000And then, because one of the things that AI needs that's so interesting is it needs enormous amounts of power.
01:09:27.000And so, just these AI centers that they're developing now, like Google's doing one where they're developing their own nuclear power.
01:09:48.000I'm usually quite conscientious with coming a little bit upset.
01:09:51.000Even when I go on the road, I've started taking a little Yeti.
01:09:55.000Coffee cup with me so I could buy coffee and just have them put it in that.
01:10:01.000The inside of those things is just lined with plastic.
01:10:04.000You're pouring hot water into plastic and then the plastic leaches into your coffee.
01:10:08.000I don't do that normally, but I figured for having a chat with you on the Jorgen show, it would be worth, I guess, some hundred microplastics.
01:10:39.000And how much does it grow exponentially in power?
01:10:42.000Because we're talking about computers, and then they start bringing up quantum computing and quantum computers' ability to do calculations, and it doesn't even make sense.
01:10:53.000And so you think, well, this is just one version of that.
01:10:57.000What is quantum computing going to look like 500 years from now?
01:11:01.000What is computing power, which is connected to AI?
01:11:05.000What is that going to look like 500 years from now?
01:11:10.000Well, we can sort of see lower bounds on what's possible, like thinking already of just the designs we can conceive of that we see in principle you could make.
01:11:20.000Maybe it'll take a long, a lot of grind to get there, but at least that establishes like a lower bound of what a technologically mature civilization could do.
01:11:28.000And then maybe they have additional ideas beyond that, but already that is enough to really unlock.
01:11:35.000So if you think of a space of possible modes of being, where like a mode of being is a way of experiencing, living, interacting, I think.
01:11:45.000You look around humanity and all people who have existed in the past, a lot of different characters, right?0.88
01:11:52.000Men, women, like mean people, nasty people, crazy people.
01:11:57.000But I think all of that diversity of human experience is like a tiny little corner in this space of possible modes of being.0.95
01:12:06.000Like it's a huge cathedral, and we've been kind of basically sitting in the janitor's closet.
01:12:11.000That's like the exploration we've done.
01:12:14.000And the kinds of Modes of being, the kinds of ways of experiencing and relating to each other or thinking and doing stuff that are ultimately possible is just this enormous space that we haven't been able to explore.
01:12:29.000Because ultimately, what we can currently conceive of and imagine and experience and feel is limited to our biological substrate, the human brain.
01:12:38.000And just as your early human ancestors from a few million years ago that you were talking about before, they wouldn't really have been able to conceive of the modern human condition.0.89
01:12:50.000Not just because I didn't think of it, but because the ape brain can't really, they don't, even if you try to describe it to them somehow, like what is music, what is humor, what is romantic love, what is science, what is all of these things, what's literature, it just doesn't fit into.0.98
01:13:12.000And so, presumably, I mean, you could think right now we've achieved the right brain size where all possible values and interesting ideas are accessible to us.0.99
01:13:23.000It seems much more likely that just as the chimpanzees are necessarily blind to a lot of what can give life meaning and value and significance, that probably are we too.
01:13:35.000Yeah, they don't even have the capacity for it, which is interesting.
01:13:39.000Yeah, we think of them as being so intelligent, and they are in comparison to a lot of other animals.
01:13:43.000But I think one of the things that they were puzzled by when they started teaching primates sign language was that they never asked questions.
01:13:53.000Like, they don't seem to have questions about stuff.
01:13:57.000So, they just exist in an intelligent way.
01:15:51.000Everybody kind of is the same for a hundred years.
01:15:54.000And inside of our lifetime, from the late 1960s for me to the early 70s for you, like in the amount of time that we've been alive, things have radically changed.
01:16:06.000Like, really, really radically to the point where it's probably the biggest shift.
01:16:11.000In human ingenuity and innovation that the world's ever seen.
01:16:18.000And we might be in the middle of the next one, which literally allows us to see what the world looks like a thousand years from now because you're going to be alive.
01:16:28.000Yeah, I mean, that's why it is now this full time job just to monitor the situation.
01:16:45.000Um, like at least in my sphere of fact, if you're doing philosophy or something like it, um, most people would think you have a kind of unlimited time horizon, and people have been working on philosophical questions for thousands of years, and um, that doesn't seem to be any huge urgency.
01:17:03.000Uh, if if if they have been unsolved for thousands of years, maybe if it takes another 500 or so, but I i always thought of philosophy as having a deadline, um, meaning that at some point.
01:17:16.000We would develop smarter than human forms of intelligence, presumably AI, that could then do the philosophy much better than us.
01:17:24.000And so there was a limited period of time during which any advances I could contribute to would be meaningful.
01:17:32.000And that it would then make sense on focusing on that subset of philosophical questions or general questions that we really need the answer to now, as opposed to like, you know, 10, 20 years in the future when somebody else can do them better, like the machine minds.
01:17:47.000So, that's kind of been a lens through which I have selected the things to work on.
01:17:51.000And now, of course, that deadline is moving closer.
01:17:57.000And so, my focus is increasingly drifting towards questions where it might be relevant to have the answer to now rather than a year or two from now.
01:18:08.000You're almost like a cultural navigator, like a guy with a sextant at the helm of a ship looking at the constellations and going, I think we're in the right direction.
01:18:19.000Yeah, but now we're sort of moving maybe into close to harbor, and you need to pay more attention to exactly how deep the rocks are.
01:18:29.000The rocks are and look, scan around you.
01:18:33.000When you think about a timeline for radical change, in your mind, what do you think that looks like?
01:18:42.000Well, I mean, I take short timelines seriously with AI.
01:18:46.000I mean, for what we know, it could be like a year or two or three or four, and probably a bit longer.
01:18:54.000But we're no longer at a point where we can be confident that we won't have super intelligence in just a few years.
01:19:04.000When you say that for the uninitiated, what do you mean by super intelligence?
01:19:08.000Well, I guess we first have AGI, artificial general intelligence, AI that can do all the stuff that we can do.
01:19:21.000And then super intelligence would just be that, but can do it way better than any human can do.
01:19:28.000So, all technical, intellectual, common sense tasks.
01:19:32.000And then we'd have robotics as well that can do all the physical stuff, not much later.
01:19:40.000And so, this, yeah, so the timeline remains uncertain, but I think it's not impossible that this could happen very soon.
01:19:49.000And then, once you have super intelligence, then I think from there on, it might be like a sort of sprint to something approximating technological maturity.
01:19:58.000Because what you have super intelligence that then designs even more smart AIs, right?
01:20:02.000Using its kind of super intelligent AI research capability and designing better chips and all of that.
01:20:09.000So, you might then have this like intelligence explosion where you go from something slightly greater than human level to some radical super intelligence that can then sort of invent whatever the remaining technologies are.
01:20:22.000Maybe there needs to be some trial and error and experiments in the physical world that slows things down a bit, but some smallish number, like a single digit number of years from super intelligence, I think you might have something that unlocks all of these like sci fi level capabilities that we've talked about.
01:20:40.000At least that seems relatively plausible to me.
01:20:48.000And my question is how does it announce itself?
01:20:52.000Does it send a mass text message to the whole world?
01:20:54.000Everybody's phone just starts like, you know, when you have those amber alerts and your phone starts vibrating, or when there's some sort of a storm warning, all of a sudden your phone goes off and then it alerts you to the fact that it's taken over.
01:22:48.000And then eventually, just more and more of the action is run by these AI systems, one or more.
01:22:54.000And they're kind of doing it at their time scale, which is speeding up.
01:23:01.000Then from that point, it would depend a lot on whether we have successfully aligned these systems so that they actually do what the people created them intended, or whether they have somehow gone off the rails.
01:23:17.000There's also the fear that America doesn't come up with it first.
01:25:04.000So the chips have gotten better, but also just the amount of funding.
01:25:07.000Like you're just building many more of these chips.
01:25:09.000And so as you apply more and more compute, like performance improves.
01:25:13.000And that's like has been a big driver.
01:25:16.000Now, at some point, like you might not be able to keep increasing the amount of money you spend on it.
01:25:23.000Because like you can go from like a thousand dollar PC to like a million dollars quite easily.
01:25:29.000And you can go from a million dollar to a billion dollars.
01:25:33.000And now, Maybe you're spending on the order of a trillion dollars across the world to build data centers per year.
01:25:40.000But you can't really do like three orders of magnitude very easily there.
01:25:43.000There's like not a thousand trillion dollars to spend on it.
01:25:46.000So at some point, just expenditure has to kind of slow down.
01:25:50.000So if we haven't achieved superintelligence by then, then maybe that would mean progress gets slower if the main driver is the scale up of compute.
01:25:58.000Now, it is also true that some of the progress is driven by algorithmic advances, like just kind of clever algorithms.
01:26:06.000But if one driver, Stalls out, then that could result in faster progress.
01:26:12.000And then, of course, there's a possibility that the people who want to pause or regulate AI gain enough traction to kind of get regulatory inhibitions.
01:26:24.000How would they do that, though, if there really is some sort of a Manhattan Project style race between the United States and China?
01:26:30.000And what other countries are developing AI right now that are close?
01:26:33.000Well, I mean, those are the two big ones.
01:26:37.000Is it possible that someone could sneak up on us and develop super intelligent AI first?
01:26:42.000Yeah, I mean, it's possible if there were like some big algorithmic.
01:26:45.000Breakthrough, for example, that made it a lot more efficient to run a similar level of capability with less sort of AI data center infrastructure.
01:26:56.000And many other countries are also trying, it's just that they are not as advanced.
01:27:01.000And out of US and China, I'd say US currently has the edge.
01:27:06.000What would happen if China got there first?
01:27:08.000Well, I mean, part would depend on whether they had successfully aligned their AI.
01:27:16.000It's unaligned, and I guess the same thing happens as if US AI is unaligned.
01:27:20.000That is, the future gets shaped according to whatever values this AI had ended up with.
01:27:27.000If the alignment problem is solved, then it might make a difference because then the values would then depend on sort of the people who owned and controlled or governed it would ask it to pursue.
01:27:42.000So then, in that scenario, maybe it makes some difference who initiates it.
01:27:50.000So, right now, the big players are, you know, there's Google and there's OpenAI, there's all these different companies.
01:27:57.000When you say aligned, do they have to all be kind of on the same page?
01:28:02.000Or like when aligned, we're not going to be aligned if there's a bunch of corporations that are competing to come up with this first.
01:28:08.000So, they have to be aligned in terms of the way they're programmed, that they're valuing human life and that they're valuing society.
01:28:47.000So, there you need, you know, political organization, appeals to the best in human nature, dialogues, like.
01:28:55.000Checks and balances, whatever stuff that might work in the political arena to hope that the governance of this, that the values to which it is aligned are sort of benign values that hopefully incorporate a wide set of stakeholders.
01:29:10.000But that's a little isn't that a little naive?
01:29:12.000Because whenever there's any sort of a situation where something has massive amounts of power above others, one of the first things they think of is what's the most money you can generate doing it and what is the best way to generate the most amount of money?
01:29:50.000Well, so that would be, I think, a lot of pressures and strains on whatever governance mechanism exists at the time when superintelligence exists.
01:30:05.000So is this something that the technology is so far ahead, the potential for it to be so far ahead of our understanding of what it's going to be able to do that, like, making laws for that now, it's going to be very difficult to even explain to people.
01:30:20.000We need these laws and we need these guardrails in place now because here's what could happen.
01:30:28.000And that conversation is not really happening right now.
01:30:41.000Well, so a few years ago, for example, there was the whole export regime imposed by the US on chips, the most advanced chips, where like China was cut out from being able to access the most advanced NVIDIA chips and so forth.
01:30:58.000And that's kind of been modified, but that was motivated to a significant extent by trying to preserve.
01:31:40.000This is the most powerful model because it seems to have significant cyber offense capabilities.
01:31:46.000It can easily detect vulnerabilities in software.
01:31:50.000And so, Anthropic figured that rather than immediately making it available to everybody, maybe it would be better to first try to make it available to providers of critical software infrastructure, like big banks, and so that they can patch up their systems.
01:32:09.000And then Fable 5 was a kind of restricted version of the Mythos model, like the same underlying model, but with extra safeguards.
01:32:21.000It basically refused anything that remotely seemed like cyber hacking, programming, biological stuff, because maybe that would be bio.
01:32:33.000So it sort of drew a wide circle around anything that even remotely looked possibly dangerous, and it just refuses that.
01:32:41.000But then after like a week or so, The administration imposed a kind of export restriction that prevented any non US citizen from using it.
01:32:55.000And that meant Anthropic had to cut it off for everybody because they didn't have a way in real time to verify who is a US citizen and not.
01:33:04.000So then it was unavailable for several weeks and intense negotiations behind the scenes and working to try to figure out because allegedly it was possible to jailbreak it so that it sometimes gave some little assistance with some cyber.
01:33:21.000And now, recently, it just became available again because they had reached some understanding with the government where it was deemed sufficiently safeguarded to be released.
01:33:33.000And now, there are efforts underway to try to work out the framework because, in the future, you wouldn't want to do this on an ad hoc basis.
01:33:41.000Like, somebody just decides this particular model for some unspecified reason.
01:33:44.000Like, you want to have clear standards, ideally, right?
01:33:47.000That applies to everybody, every company.
01:33:49.000So, there's now some industry wide effort to try to work with the government to.
01:33:53.000Anyway, so there's like a lot of this stuff happening.
01:33:56.000I expect much more of this going forward.
01:33:59.000This going forward, it just has recently become like a serious issue.
01:34:05.000People until recently were kind of ignoring the whole AI thing for the most part.
01:34:10.000And then there is a second wave coming, I think.
01:34:14.000So far, we haven't really seen any big impact on the labor market from AI.
01:34:20.000But once that starts to hit, and you get maybe high levels of unemployment amongst white collar workers, and imagine if you have millions and millions of people with.
01:34:32.000That have their university diplomas, right?0.53
01:34:34.000They feel a sense of superiority and entitlement.
01:34:36.000They've gone through the whole process.
01:34:38.000They got their degree and now they expect a well paying job.
01:34:45.000They've got nothing to do all day long but complain about AI.
01:34:50.000So you're going to have all these well educated people who feel resentful and are going to say every possible bad thing about AI that could be said all day long and mobilize.
01:34:58.000They're a very powerful political constituency that will emerge from that.
01:35:03.000That's not even yet happened, but that will kind of add to all of these other.
01:35:08.000Grievances that people point to with AI.
01:35:11.000So I think there's going to be kind of significant political pressures for doing something about AI.
01:35:19.000Isn't the key getting ahead of it though?
01:35:21.000So, how can people find, how can we see the vulnerabilities in advance and recognize like when this is going to, like if there is going to be a tipping point and a bunch of white collar workers are going to be out of work and there's going to get to a point where we realize like this is coming.
01:35:56.000I mean, I've been thinking about this for like three decades, maybe, and I still feel extremely unsure even which direction is kind of up and which is down.
01:37:15.000Well, I think it's eventually going to get.
01:37:17.000To wherever it's going to get, either way.
01:37:20.000Having it slower, I don't know if that's really going to help us.
01:37:24.000I think almost like we have to crash and then we have to figure out how to rebuild and pick up the pieces.
01:37:30.000I don't think we're going to be intelligent or have enough foresight to recognize where all these flaws and where all these problems are going to ultimately be.
01:37:40.000I think they're just going to have to happen and then people are going to have to adjust.
01:37:46.000So I'm not advocating an AI pause by contrast to a lot of.
01:37:50.000Some of my colleagues and friends and stuff.
01:37:54.000I could see some scenario in which it would be helpful at some point to have a temporary slowdown of a few months, maybe or half a year or a year.
01:38:04.000If you imagine there are different companies, countries maybe racing to get there first.
01:38:10.000And then eventually somebody figures out they basically have the system in place.
01:38:14.000They just need to amp up, run it for longer.
01:38:17.000They can see that it will become super intelligent.
01:38:22.000It might be very helpful in that situation to have a few extra months just to sort of double check all your safeguards.
01:38:29.000And rather than immediately cranking all the knobs up to 11, like maybe sort of do it a little bit incrementally, watch what happens, study it, then crank it up a little bit more.
01:38:39.000I just think you might gain some extra safety if you have a few extra months there.
01:38:42.000Like the pressures on these people in the lab.
01:38:55.000Like, even just being able to sleep properly for like between your work sessions and like having, you know, a weekend to mull things over, like just that kind of human, humans don't really operate on, on, well, on, in, in these like super short timelines.
01:39:11.000So, I think a short, if you could be sure that the pause would, in fact, be limited and short and then it would be lifted, I think that could be quite useful to have at, if it could also be timed to be, At the right moment and well implemented.
01:39:26.000But wouldn't the problem be espionage, first of all?
01:39:31.000Yeah, if somebody realized, hey, they're about three days away from this, and then China bribes a bunch of people, and people take off and move to.
01:39:42.000Yeah, there are various downsides to passing.
01:39:47.000So one is that, and there are competitors who just even without espionage are catching up.
01:39:55.000Another is it's not as if the world is.
01:40:02.000There are other, like, whether the natural hazards, I think they are very small, like super volcanoes and stuff like that on the relevant time scales.
01:40:32.000Empirics of these natural catastrophes.
01:40:34.000By contrast, I would say there are other hazards that are not low.
01:40:39.000I mean, we have rapid advances in synthetic biology, some of them driven or enabled maybe even by already AI progress that has already taken place, like that can start to assist with this, and more just happening independently.
01:40:56.000So you get increasing risks from synthetic biology.
01:40:59.000We still have the nuclear arsenals, right, kind of existing.
01:41:03.000Um, we, I think, have gotten a little bit complacent about the risks of nuclear war, um, and various other things as well.
01:41:15.000So, I think there's like a background level of existential risk that humanity faces in the absence of superintelligence that probably is growing as well.
01:41:24.000So, you don't want to wait so long that you don't even get the chance to roll the dice with AI because you destroyed yourself before you even got an AI.
01:41:35.000And then also, I mean, I think there is like some risk if you set up a pause infrastructure that what's initially meant to be a temporary thing becomes permanent.
01:41:47.000Like they say, there's nothing more permanent than a temporary government program.
01:41:51.000So, if you set up the infrastructure to actually control this, ideally at a global scale, right, it's meant to be for six months.
01:42:39.000If there is something that somehow could have a chance to fix this, like all the suffering that is happening in the world, aside from the people dying, like all the people who are bereaved, right?
01:42:52.000Their loved one, and then all the disease that led up to that dying.
01:42:57.000And then all the non disease related, all the horrible poverty and like the suicidal depression and the animals in the animal factories that they're like spending.
01:43:07.000There's just like this, I think, moral urgency that if there is a hope that getting AI right could fix this, then you don't want to wait unnecessarily long because every day is just this massive horror.
01:43:28.000But there is a trade off because if we can make more progress on AI safety and alignment and get our act together a little bit before we take the plunge, that's also worth quite a lot.
01:43:41.000It's certainly exciting because the possibilities are right in front of us and they're kind of endless.
01:43:49.000And it seems like they're right in front of us.
01:43:50.000It seems like, in my mind, it seems like we're like 24 months away from something really insane.
01:43:56.000I think it could be, could be, could be 48 months.
01:43:59.000It could be, and we don't, we don't, yeah, right, it could be, yeah, who knows.
01:44:18.000And, um, one day, cat's going to be out of the bag, and there's not going to be any way to stop it.
01:44:24.000Like, if the power went out right now, if there was some sort of a massive solar flare that.
01:44:29.000Killed our power grids and all computer hard drives got fried, and we had a restart from now.
01:44:38.000We would have to rebuild society, right?
01:44:40.000But if we get to the point where this thing understands what would cause that, how to prevent that, how to make sure that never happens, much better power supply, much better allocation of resources, much better batteries, much better redundant data systems where you never have to worry about.
01:45:01.000Hard drives crashing, and you never have to worry about any of these problems.
01:45:05.000Like, any information you have now will be secure, and then understanding of natural disasters will actually get drilled into this volcano and it never goes off.
01:45:14.000Like, there's going to be a few things that they're going to figure out through AI that's going to prevent a lot of the things that have probably wiped out enormous swaths of people.
01:45:21.000I mean, imagine if we could just actually see asteroids coming, all of them, and know how to divert them instantaneously.
01:45:29.000We have a bunch of ideas right now on how to do it, right?
01:45:43.000There's a bunch of different crazy ideas, but imagine if AI is like, actually, you could just do this.
01:45:49.000And then you put a shield over the Earth and you never have to worry about it ever.
01:45:53.000The shield's powered by the sun, so you have ultimate power and you never have to worry about being hit by a Manhattan sized asteroid.
01:46:01.000I think the trickier ones might be ones that are more internal to civilization.
01:46:07.000If you have some process that's either like a worldwide process with different humans and corporations and governments or like an internal process, Process in the AI.
01:46:19.000On the one hand, you want to be able to continue to develop new ideas, new ways of doing things.
01:46:26.000So you need to experiment and try new things.
01:46:28.000But then there's also the risk of just keep trying new ideas, like that, eventually you get stuck on some idea that actually proves really harmful or dangerous to you.
01:46:40.000And that you then sort of derail internally through your own memetic or internal evolutionary development.
01:46:49.000How to sort of grow up safely in this world where you have unlocked all kinds of new technological capabilities.
01:46:58.000Even if you're easily able to protect yourself against external threats like volcanoes and asteroids and stuff, right?
01:47:04.000There might still be processes arising from within this global civilization.
01:47:10.000Just as we might invent new technologies that are dangerous, we might invent new drugs that are super addictive.
01:47:17.000Once you invent them, you try them and then you're addicted.
01:47:28.000Because that's one of the major fears that a lot of people have is that human beings will develop ideologies based around AI, that there'll be like an AI god that someone develops.
01:47:41.000And I mean, I think that's, in fact, I would add that to the list of, I mentioned nuclear and bio terrorist risk as like background existential risks.
01:48:40.000There's algorithms, there's AI, there's people that are hired to do it, there's people that are working for certain organizations that are hired to muddy up the waters, gaslight people, create problems, have people argue with each other.
01:48:55.000If AI recognizes how easy it is to manipulate people, do whatever it wants, that's a fear as well.
01:49:04.000So, whenever you change the basic parameters of sort of the social, cultural, political discourse, new dynamics will emerge.
01:49:18.000And we don't have the kind of social science that is able to predict what happens if you change some of these knobs.
01:49:23.000So, we've seen in the past, like you invent, let's go all the way back, like somebody invents writing.
01:49:29.000Okay, so that turns out to have had a huge effect, not just on people writing literature and stuff, but on political systems.
01:49:37.000You could now have states that could keep tax records, right?
01:49:41.000So, you could have larger political units with writing, and then they can hire standing armies.
01:49:47.000And now you have large scale war, you have social stratification, you could have the ruler of a large area have enormous wealth, and you could have a soldier caste that protects against internal and external.
01:50:00.000So, just the way that human societies are politically organized changes as a result of this change in the rules of communication when you can have written texts.
01:50:10.000And then you have the printing press again, with like 100 years.
01:50:14.000Of religious wars in Europe, like Reformation and all of this stuff, is possibly.
01:50:19.000And then, like, forms of democracy later on, also coming out of this, and the scientific revolution.
01:50:27.000Then you have, like, mass media in the last century with, like, radio and stuff, and you have kind of demagogues that take advantage of being able to simultaneously talk to millions of people, because that was never possible before, right?
01:50:38.000You could have some charismatic kind of guy who's, like, rallies up the whole nation, and then new ideologies become.
01:50:45.000Mimetically fit in that situation that might never have, if it were people writing kind of letters to the editor, like it's kind of a different type of idea that works there than if you're giving a kind of stump speech that goes out to millions simultaneously.
01:50:59.000And now with social media, of course, we have another one of these, and you do see that starting to change culture in different ways.
01:51:07.000And now with AI being the next wave of this, that will also change.
01:51:10.000You have like bots, you have new ways of finding information, you have maybe AIs that can themselves be super persuaders.
01:51:18.000Presumably, in some unknown, unknowable way, the way that social discourse pans out.
01:51:25.000And for any one of these, I guess it could turn out to be a lot better.
01:51:28.000We could become more informed having AI advisors.
01:51:30.000I think that's a fairly likely scenario, but it's also possible somehow the dynamics shape out in a different way and we kind of go collectively insane in some way.
01:51:43.000And it all becomes kind of totalitarian or we sort of fragment into like political warring tribes or.
01:51:51.000We become kind of completely unhinged.
01:51:53.000Every one of us becomes convinced of their own little nutty theory that they then, like their AI feed, just serves them more material to kind of fuel their conviction.
01:52:02.000There are different ways in which this could go badly.
01:52:07.000But don't you think even if that happens, ultimately the progress of AI won't stop?
01:52:13.000And so, again, I keep going back to this thing, but I think this is really what it is we have to change.
01:52:23.000All those things are only problems if people stay what we are now, which is territorial primates with desire to possess material goods for some strange reason, even though we're a finite life form.
01:52:36.000If all that changes, if we change what it means to be a person, which seems inevitable, then it won't matter.
01:52:48.000If we could figure out a way to literally engineer out.
01:52:53.000All of the issues that humans have with greed and violence and all the different things that trouble society, if the desire for that is no longer a part of being a person, which is that's doable.
01:53:06.000That seems like if we're going to continue to evolve past, you know, Australia Pithecus to Homo sapien 2026, if we continue the same amount of time in the future, we probably won't be like that anymore.
01:53:20.000The best versions of people aren't people that want to steal your house and steal your land and shoot you and take your resources.
01:53:27.000The best people are the people that contribute and they're interesting and they make you excited to be around them and you like it.
01:53:34.000If that keeps going on and if that is aided by technology, if we recognize that there's actual patterns of human thinking and behavior that can be changed, and that if we all agree to subscribe to this algorithm, to connect to this computer program, connect to this external device or maybe not even external, maybe it's internal that allows us to communicate with each other telepathically.
01:54:01.000Like all that changes, and then it doesn't matter.
01:54:06.000It doesn't matter if there's any guardrails for AI or not because we're not the same thing anymore.
01:54:10.000Like all the problem, the worry that we have about AI, the worry we have about power and manipulation and the ability to influence people, all that kind of goes away if people don't behave the way they behave currently.
01:54:25.000Well, I guess here's one difference with your like white water rafting metaphor.
01:54:44.000But I think in reality, what might also be the case is that in addition to trying to not smash ourselves on the rock, there might be places where the choices we make affect the ultimate destination.
01:55:04.000And where, like, If you steer in one direction, you might end up in one place ultimately, in some sort of strange post human world, and maybe it's really beautiful and people have the chance to grow into their true self and we love each other and are creative and take initiatives and go a different way.
01:55:21.000Maybe you have a paperclip maximizing AI area, maybe that just everything is paperclip.
01:55:26.000So going a third way, maybe you have like a totalitarian system with like a small elite on top and everybody else.
01:55:37.000And there are many more possibilities and maybe some that.
01:55:39.000Kind of, we would think would look kind of good if we chose now, but then in reality, they have some hidden flaw that we didn't think of.
01:55:48.000So, if we chose those, it would sort of be a mistake.
01:55:50.000And then maybe others that don't look that appealing to us now, if you just presented them in a brochure to you, like, but then actually, if you thought hard about it, maybe you would realize that actually would be a really nice place to live.
01:56:02.000Because, like, in the current world, there are places that are nice to visit that are interesting, but then those are not necessarily the same places we would want to live and raise.
01:56:11.000So there are like novels and movies that are really fun to watch, but you wouldn't want to live in those worlds.
01:56:17.000So there's a difference between the kind of what makes for the good, interesting story and the place we actually want to spend the rest of your life.
01:56:23.000And so I think there would be possibly a lot of opportunities to make foolish choices or unwise choices or conflicts that sort of thwart the process and make us end up not just that we go extinct before we reach there, but that the deer that we reach might depend on.
01:56:40.000The level of wisdom and kindness that we have during the process.
01:56:44.000And there's always unintended consequences with every solution that we try to find for any problem that we have.
01:56:50.000There's always some new thing that comes up.
01:56:51.000We're like, oh, we didn't see that coming.
01:56:53.000I mean, certainly whenever we're dealing with nature, like whenever they've brought in invasive species to handle other invasive species, it's always a giant disaster, you know?
01:57:48.000So they had millions of these things that were like all over the highway.
01:57:51.000So you drive on them, you just squash them.
01:57:53.000Everywhere you go, because there was nothing controlling the population, these things anymore.
01:57:58.000It's like unintended consequences of, you know, because we're very short sighted in even our ability to be contemplative about what the possibility of the future is when you're dealing with such an open ended thing like AI, super general intelligence that can maybe make better versions of itself.
01:58:18.000Like, who the hell knows what that means?
01:58:23.000Because it seems like if it keeps going and that, if it makes better versions of itself, what's ultimately going to get To the point where it could do anything, which is exactly what a god is.
01:59:03.000We're not really, I mean, you have to still try to make your best, right?
01:59:06.000But then add to that that all the passengers are fighting amongst each other.
01:59:09.000Each one thinks they are the guy who should control the, like, they're all convinced that they are the superior person to try to land the plane.
01:59:19.000So, in addition to trying to figure out how the controls work, they are like actually having a big fist fight in the cockpit as well.
01:59:25.000And somebody's like dragging somebody else away.0.88
01:59:27.000And there's like the kind of monk tribe that is the current state of the world here.0.62
01:59:51.000At some point, we should be able to hand over the reins.
01:59:57.000Once you have a sufficiently good AI, we maybe get assistance from that point on.
02:00:02.000I think the first thing that it's going to take over is government, who's going to realize how.
02:00:08.000Unbelievably inefficient the government is at doing almost everything, and how much of the money that gets allocated is fraud and waste.
02:00:18.000If you allow AI to sort through that and develop much more efficient pathways to controlling and disseminating resources, some people would not want to see that particular thing happen.
02:00:30.000But if we get to the point where there's some sort of a hive mind possibility, some sort of a I mean, one of the things that Elon said that I thought was really fascinating, he said, You're going to be able to talk without words.
02:00:45.000Well, if we're able to talk without words, does that eventually get to the point where we could read minds?
02:00:51.000Is thought and is communication no longer verbal?
02:00:59.000So, right now, we associate sounds that I'm making where you know what words I'm using, what I'm referencing, and we get a certain understanding of what each other is trying to say.
02:01:09.000Well, what if that's just clunky and that's silly?0.99
02:01:36.000Because right now, what we figured out is sort of like duct tape.
02:01:41.000We're communicating, we've kind of patched it up, we figured out something.
02:01:45.000We're just going to use noises we make.
02:01:47.000Well, we have different noises here than the people that live on this island.
02:01:50.000They have totally different noises.0.99
02:01:50.000I don't know what the fuck they're saying.0.99
02:01:52.000And then you have people on the other side of the world, totally different noises.0.99
02:01:55.000So, Tower of Babel type situation, right?
02:01:58.000Where we really can't communicate with each other unless we have translators.
02:02:01.000But if we get to the point where that's not how humans communicate, we communicate purely through thought and intention and understanding, and that it's no longer based on language.
02:02:13.000It's no longer based on this is a transistor, this is a coffee mug.
02:02:18.000Instead, It's a complete understanding of each individual thing that we're discussing.
02:02:31.000I mean, I guess the cyber security implications are significant if you are giving direct access to other people to transmit signals to your brain in a high bandwidth way that is not just words kind of that it's almost like you heard them even though there's no sound in your ear, but if it's like actually directly kind of interfacing in a high bandwidth way with your.
02:04:14.000That's where you probably would want encryption, right?
02:04:16.000If you're going to transmit thoughts to me through something, you don't want to have a.0.99
02:04:21.000Oh, you don't want some asshole constantly in your head, your next door neighbor, just poking you and prodding you.0.98
02:04:27.000But you would hope that along with this technology becomes like a general state of enlightenment.0.99
02:04:33.000That the human beings achieve, where that's no longer the kind of behavior that we indulge in.
02:04:40.000Which behavior do we no longer indulge in?
02:04:42.000Annoying each other, getting each other out, stealing each other's money, that kind of stuff.
02:04:45.000Well, but then, yeah, I guess it goes back to this question of the utopian condition.
02:04:50.000So, there are a lot of things that individually are bad, like lying, stealing, cheating, greed, excessive pride, all kinds of disease, stubbing your knees bad.
02:05:07.000Yet, if you imagine removing all of those things, then that changes the human condition quite profoundly.
02:05:14.000And to some people, it would feel kind of maybe or appear flavorless or sort of if there is.
02:05:22.000No tension, no conflict, no bruised ego, no like.
02:05:27.000But it might still be good, but it does force us to sort of.
02:05:31.000I think it would be a rather fundamentally different thing that we would be metamorphosing into if we went all the way in that direction, which ultimately might be right.
02:05:38.000But it would require us to kind of find new ways of realizing whatever values are imperfectly realized in the current world through conflict and competition and pain.
02:05:49.000Like some people get the motivation from painful failure.
02:05:53.000And so, if you get rid of the pain from failure, then you'd need some other motivation, like some other thing that drives you on, which there could be.
02:06:01.000Like maybe it's just a love of achievement and you feel kind of neutral or just less happy when you fail.
02:06:06.000But you would still need something that kind of preserves whatever structure it is that we think is valuable in the current human condition.
02:06:15.000Unless you go all the way to sort of radical hedonism and think the future is best if we were all just kind of floating in some kind of drug induced euphoria as blobs that.
02:06:27.000Experienced immense pleasure, but had no real texture in our experiences, didn't engage in activities and didn't interact with each other.
02:06:36.000Like, there's a philosophical view where ultimately pleasure is the only thing that matters and the minimization of suffering.
02:06:42.000So, if that's your axiology, then it's relatively easy to see how at technological maturity you would achieve a sort of optimal state.
02:06:50.000But if you have a more complex value system where maybe pleasure is one good thing, maybe really important, But there are also other things like appreciating beauty, you know, true friendship, courage, achievement.
02:07:03.000And ideally, you'd want a future that includes all of these things.
02:07:06.000Then you need to do a little bit more sort of design work to figure out a way to combine them all in a meaningful way.
02:07:14.000This all comes back to our idea of human meaning, what's important to humans, our finite 100 year lifespan adoption of this concept of meaning.0.94
02:07:24.000But the black hole doesn't give a shit about human meaning.0.96
02:07:27.000And it's going to be around a lot longer than us.0.99
02:09:05.000And so, the future would consist, I think, if we succeed, in a lot of game playing.
02:09:11.000And you could certainly have these artificial purposes that you set yourself, these goals that then give you a reason to engage in an activity.
02:09:17.000Now we're back in the world of Half-Life.
02:09:25.000This is going to be your artificial goal.
02:09:27.000And you could imagine, I think maybe we shouldn't think of video games here, but it could be much like games we can't even imagine.
02:09:34.000It could be like society-wide games that last for 20 years that involve all kinds of multimodal things and little groups that work together to come up with new ways of creatively.
02:09:49.000In that broad sense of kind of things we do for their own sake, I think game playing could be, and it's like a lot of what children do, they're kind of for curiosity and spend a lot of their time playing games, and we might all be like kids again.
02:10:00.000What might be in shorter supply is sort of natural purpose, like purposes which we don't just arbitrarily make up in order to have a purpose, but that are sort of given to us.
02:10:14.000So, right now in the world, you might say, you know, making a living.
02:10:20.000It's not just an arbitrary purpose because there are real consequences if you fail.
02:10:24.000Like maybe eventually you get kicked out from your flat and then it's really cold and you get rained on and horrible things happen.
02:10:33.000So, similarly, if you don't brush your teeth, eventually you will have tooth decay and there will be real consequences.
02:10:40.000So, there are various things that you have reason to do because there are real negative consequences if you fail to do them.
02:10:46.000And a lot of our lives is structured by these natural purposes.
02:10:50.000At a societal level, there's a whole bunch of things we need to do together, right?
02:10:55.000In this future world, maybe there would be many fewer of those natural purposes, because for any one of them, you could just ask the AI to sort it out.
02:11:02.000And so the artificial purposes would be a larger chunk.
02:11:04.000It's interesting to think are there any natural purposes that would survive to technological maturity?
02:11:10.000Like any things that we still have sort of instrumental reasons that we need to do ourselves?
02:11:17.000And I think there might be a few, but they are more subtle.
02:11:20.000They might not strike us currently as very.
02:11:23.000Important, but it's one of those things where, like, you know, that during the day, if you're outside, you can't see the stars, right?
02:11:35.000It's not because they're not there, like, it's because, like, there's so much light that they are sort of blotted out.
02:11:40.000But at night, when the stronger light from the sun is absent, you can see these fainter lights.
02:11:47.000I think similarly, in this future, there might be once this sort of urgent screaming moral values.
02:11:54.000Of immediately pressing practical concern go away, you might be able to perceive a whole constellation of these more subtle values that we are blind to currently.
02:12:02.000So, take the value of like, I don't know, like honoring your forebears.
02:12:09.000So, right now, it doesn't seem, I mean, maybe it's nice sometimes to remember your past parents or some historical hero who did something good that benefited humanity, right?
02:12:21.000But it's like not the main thing that you're like, maybe that would be a bigger thing if there was nothing else you needed to do.
02:12:26.000Maybe you could actually spend serious time.
02:12:31.000Like, even for people who are very religious, a lot of their actual waking hours are spent on random other things, doing their laundry, right?
02:12:39.000Like driving to work, like all kinds of stuff.
02:12:41.000Like, if all of that was automated, you could imagine spending more time on trying to align yourself, orient yourself to this higher being, and trying to be in communication with them.
02:12:56.000Like, there may be some things that would just be kind of nice and cool if the world were like that.
02:13:01.000We don't really have time to worry so much about them now, but if there was nothing else on the agenda, like coming together in a way that upholds some tradition in a beautiful, original way that's Still is true to the original spirit together with other people and enacting some ceremony.
02:13:17.000Like maybe those things would start to fill more of our time in conjunction with this game playing.
02:13:25.000And there might be many other of these kind of subtler values that would start to shape what people were doing.