I feel that AI should be built by humanity for humanity, and let’s not forget that. — Max Tegmark

“The basic problem is you just don’t want to give, you don’t wanna cede control over your planet to some other more intelligent entity that doesn’t share your goals. It’s that simple.” — Max Tegmark

LEX: But even with time, the AI alignment problem, it seems to be really difficult. MAX: Oh yeah. But it’s also the most worthy problem, the most important problem for humanity to ever solve. Because if we solve that one, Lex, that aligned AI can help us solve all the other problems.” 

“The one thing which is so obvious though, which, I think is just really worth reflecting on, is because the mind space of possible intelligences is so different from ours, it’s very dangerous if we assume they’re gonna be like us, or anything like us.” — Max Tegmark

LEX. I mean, that’s one possible hopeful trajectory here, is that humans will continue to human, and AI will just be a kind of, a medium that enables the human experience to flourish.

MAX. Yeah, I would phrase that as rebranding ourselves from Homo sapiens to Homo sentiens. You know, right now, if it’s sapiens, the ability to be intelligent, we’ve even put it in our species’ name. So we’re branding ourselves as the smartest information processing entity on the planet. That’s clearly gonna change if AI continues ahead. So maybe we should focus on the experience instead, the subjective experience that we have, Homo sentiens, and say that’s what’s really valuable, the love, the connection, the other things, and get off our high horses, and get rid of this hubris that, you know, only we can do integrals. – So consciousness, the subjective experience is a fundamental value to what it means to be human. Make that the priority. – That feels like a hopeful direction to me. But that also requires more compassion, not just towards other humans, because they happen to be the smartest on the planet, but also towards all our other fellow creatures.

LEX. So you’ve mentioned to me that there’s an open letter that you’re working on.

MAX. It’s actually going live in a few hours. (Lex laughing) So I’ve been having late nights and early mornings. It’s been very exciting, actually. In short, have you seen, “Don’t Look Up,” the film? – Yes, yes. – I don’t want to be the movie spoiler for anyone watching this who hasn’t seen it. But if you’re watching this, you haven’t seen it, watch it, because we are actually acting out, it’s life imitating art. Humanity is doing exactly that right now, except it’s an asteroid that we are building ourselves. Almost nobody is talking about it. People are squabbling across the planet about all sorts of things, which seem very minor compared to the asteroid that’s about to hit us, right? Most politicians don’t even this on the radar, they think maybe in 100 years or whatever. Right now we’re at a fork on the road. This is the most important fork that humanity has reached in it’s over 100,000 years on this planet. We’re building effectively a new species that’s smarter than us, it doesn’t look so much like a species yet ’cause it’s mostly not embodied in robots. But that’s the technicality which will soon be changed. And this arrival of of artificial general intelligence that can do all our jobs as well as us, and probably shortly thereafter, superintelligence, which greatly exceeds our cognitive abilities. It’s gonna either be the best thing ever to happen to humanity or the worst. I’m really quite confident that there is not that much middle ground there. – But it would be fundamentally transformative to human civilization. – Of course, utterly and totally. Again, we’d branded ourselves as Homo sapiens ’cause it seemed like the basic thing, we’re the king of the castle on this planet, we’re the smart ones, we can control everything else, this could very easily change. We’re certainly not gonna be the smartest on the planet for very long if AI, unless AI progress just halts, and we can talk more about why I think that’s true ’cause it’s controversial. And then we can also talk about reasons we might think it’s gonna be the best thing ever, and the reason we think it’s going to be the end of humanity, which is of course, super controversial. But what I think we can, anyone who’s working on advanced AI can agree on is, it’s much like the film “Don’t Look Up,” in that it’s just really comical how little serious public debate there is about it, given how huge it is.

LEX. So what we’re talking about is a development, of currently, things like GPT-4, and the signs it’s showing of rapid improvement that may, in the near term lead to development of superintelligent AGI, AI, general AI systems, and what kind of impact that has on society.

MAX. Exactly.

LEX. When that thing achieves general human-level intelligence, and then beyond that, general superhuman level intelligence. There’s a lot of questions to explore here. So one, you mentioned halt. Is that the content of the letter? is to suggest that maybe we should pause the development of these systems.

MAX. Exactly, so this is very controversial, from when we talked the first time, we talked about how I was involved in starting the Future of Life Institute, and we worked very hard on 2014, 2015, was the mainstream AI safety. The idea that there even could be risks and that you could do things about them. Before then, a lot of people thought it was just really kooky to even talk about it. And a lot of AI researchers felt, worried that this was too flaky, and could be bad for funding, and that the people had talked about it were just not, didn’t understand AI. I’m very, very happy with how that’s gone, and that now, you know, it’s completely mainstream, you go in any AI conference, and people talk about AI safety, and it’s a nerdy technical field full of equations and blah-blah.

LEX. Yes.

MAX. As it should be, but there is this other thing, which has been quite taboo up until now, calling for slowdown. So what, we’ve constantly been saying, including myself, I’ve been biting my tongue a lot, you know, is that, we don’t need to slow down AI development. We just need to win this race, the wisdom race between the growing power of the AI and the growing wisdom with which we manage it. And rather than trying to slow down AI, let’s just try to accelerate the wisdom, do all this technical work to figure out how you can actually ensure that your powerful AI is gonna do what you want it to do. And have society adapt also with incentives and regulations so that these things get put to good use. Sadly, that didn’t pan out. The progress on technical AI capabilities has gone a lot faster than many people thought back when we started this in 2014. Turned out to be easier to build really advanced AI than we thought. And on the other side, it’s gone much slower than we hoped with getting policymakers and others to actually put incentives in place to make, steer this in the good directions, maybe we should unpack it and talk a little bit about each, so.

LEX. Yeah. – Why did it go faster than a lot of people thought? – Can you briefly, if it’s just a small tangent, comment on your feelings about GPT-4? So just that you’re impressed by this rate of progress, but where is it? Can GPT-4 reason? What are like the intuitions? What are human interpretable words you can assign to the capabilities of GPT-4 that makes you so damn impressed with it? 

MAX. I’m both very excited about it and terrified. It’s interesting mixture of emotions. (laughs) – All the best things in life include those two somehow. – Yeah, it can absolutely reason, anyone who hasn’t played with it, I highly recommend doing that before dissing it. It can do quite remarkable reasoning. I’ve had to do a lot of things, which I realized I couldn’t do that myself that well even, and it obviously does it dramatically faster than we do too, when you watch it type, and it’s doing that well, servicing a massive number of other humans at the same time. The same time, it cannot reason as well as a human can on some tasks, it’s obviously the limitations from its architecture. You know, we have in our heads, what in geek-speak is called a recurrent neural network. There are loops, information can go from this neuron, to this neuron, to this neuron, and then back to this one, you can like ruminate on something for a while, you can self-reflect a lot. These large language models, they cannot, like GPT-4. It’s a so-called transformer where it’s just like a one-way street of information, basically. In geek-speak, it’s called a feed-forward neural network. And it’s only so deep, so it can only do logic that’s that many steps and that deep, and it’s not, so you can create problems which it will fail to solve, you know, for that reason. But the fact that it can do so amazing things with this incredibly simple architecture already, is quite stunning, and what we see in my lab at MIT when we look inside large language models to try to figure out how they’re doing it, which, that’s the key core focus of our research, it’s called mechanistic interpretability in geek-speak. You know, you have this machine that does something smart, you try to reverse engineer it, and see how does it do it. I think of it also as artificial neuroscience, (Lex laughs) ‘Cause that’s exactly – I love it. – what neuroscientists do with actual brains. But here you have the advantage that you can, you don’t have to worry about measurement errors. You can see what every neuron is doing all the time, and a recurrent thing we see again and again, there’s been a number of beautiful papers quite recently by a lot of researchers, and some of ’em are here even in this area, is where when they figure out how something is done, you can say, “Oh man, that’s such a dumb way of doing it.” And you read immediately see how it can be improved. Like for example, there was this beautiful paper recently where they figured out how a large language model stores certain facts, like Eiffel Tower is in Paris, and they figured out exactly how it’s stored and the proof of that they understood it was they could edit it. They changed some synapses in it, and then they asked it, Where’s the Eiffel Tower?” And it said, “It’s in Rome.” And then they asked, “How do you get there? Oh, how do you get there from Germany?” “Oh, you take this train, the Roma Termini train station, and this and that,” “And what might you see if you’re in front of it?” “Oh, you might see the Colosseum.” So they had edited, – So they literally moved it to Rome. – But the way that it’s storing this information, it’s incredibly dumb, if any fellow nerds listening to this, there was a big matrix, and roughly speaking, there are certain row and column vectors which encode these things, and they correspond very hand-wavingly to principle components and it would be much more efficient for as far as matrix, just store in the database, you know and, and everything so far, we’ve figured out how these things do are ways where you can see it can easily be improved. And the fact that this particular architecture has some roadblocks built into it is in no way gonna prevent crafty researchers from quickly finding workarounds and making other kinds of architectures sort of go all the way, so. In short, it’s turned out to be a lot easier to build close to human intelligence than we thought, and that means our runway as a species to get our shit together has has shortened. – And it seems like the scary thing about the effectiveness of large language models, so Sam Altman, I’ve recently had conversation with, and he really showed that the leap from GPT-3 to GPT-4 has to do with just a bunch of hacks, a bunch of little explorations with smart researchers doing a few little fixes here and there. It’s not some fundamental leap and transformation in the architecture. 
LEX. And more data and more compute. 
MAX. And more data and compute, but he said the big leaps has to do with not the data and the compute, but just learning this new discipline, just like you said. So researchers are going to look at these architectures and there might be big leaps where you realize, “Wait, why are we doing this in this dumb way?” And all of a sudden this model is 10x smarter. And that that can happen on any one day, on any one Tuesday or Wednesday afternoon. And then all of a sudden you have a system that’s 10x smarter. It seems like it’s such a new discipline, it’s such a new, like we understand so little about why this thing works so damn well, that the linear improvement of compute, or exponential, but the steady improvement of compute, steady improvement of the data may not be the thing that even leads to the next leap. It could be a surprise little hack that improves everything. – Or a lot of little leaps here and there because so much of this is out in the open also, so many smart people are looking at this and trying to figure out little leaps here and there, and it becomes this sort of collective race where, a lot of people feel, “If I don’t take the leap someone else will,” and it is actually very crucial for the other part of it, why do we wanna slow this down? So again, what this open letter is calling for is just pausing all training of systems that are more powerful than GPT-4 for six months. Just give a chance for the labs to coordinate a bit on safety, and for society to adapt, give the right incentives to the labs. ’cause I, you know, you’ve interviewed a lot of these people who lead these labs and you know just as well as I do that they’re good people, they’re idealistic people. They’re doing this first and foremost because they believe that AI has a huge potential to help humanity. But at the same time they are trapped in this horrible race to the bottom.

MAX. and had to rein it in, you know, in a big way. I think once this basic message comes out that this isn’t an arms race, it’s a suicide race, where everybody loses if anybody’s AI goes out of control, it really changes the whole dynamic. It’s not, and I’ll say this again ’cause this is this very basic point I think a lot of people get wrong. Because a lot of people dismiss the whole idea that AI can really get very superhuman because they think there’s something really magical about intelligence such that it can only exist in human minds, you know, because they believe that, they think it’s gonna kind of get to just more or less “GPT-4 plus plus,” and then that’s it. They don’t see it as a suicide race. They think whoever gets that first, they’re gonna control the world, they’re gonna win. That’s not how it’s gonna be. And we can talk again about the scientific arguments from why it’s not gonna stop there. But the way it’s gonna be, is if anybody completely loses control and you know, you don’t care if someone manages to take over the world who really doesn’t share your goals, you probably don’t really even care very much about what nationality they have, you’re not gonna like it much worse than today. If you live in Orwellian dystopia, what do you care who’s created it, right? And if someone, if it goes farther, and we just lose control even to the machines, so that it’s not us versus them, it’s us versus it. What do you care who created this unaligned entity which has goals different from humans, ultimately? And we get marginalized, we get made obsolete, we get replaced. That’s what I mean when I say it’s a suicide race, it’s kind of like we’re rushing towards this cliff, but the closer the cliff we get, the more scenic the views are, and the more money there is there, and the more, so we keep going, but we have to also stop at some point, right? Quit while we’re ahead, And it’s, it’s a suicide race which cannot be won, but the way to really benefit from it is, to continue developing awesome AI a little bit slower. So we make it safe, make sure it does the things that humans want, and create a condition where everybody wins. The technology has shown us that, you know, geopolitics and politics in general is not a zero sum game at all.

Yeah, but it seems like this particular technology has gotten so good so fast, become powerful to a degree where you could see in the near term, the ability to make a lot of money. – [Max] Yeah. – And to put guardrails, to develop guardrails quickly in that kind of context seems to be tricky. It’s not similar to cars or child labor, it seems like the opportunity to make a lot of money here very quickly is right here before us. – So again, there’s this cliff. – Yeah, it gets quite scenic, (laughs) – [Max] The closer to the cliff you go, – Yeah. – The more money there is, the more gold ingots there are on the ground you can pick up or whatever, if you want to drive there very fast, but it’s not in anyone’s incentive that we go over the cliff and it’s not like everybody’s in the wrong car.

MAX. And then we are giving ever more power to things which are not alive. These large corporations are not living things, right? They’re just maximizing profit. I wanna win the war on life. I think we humans, together with all our fellow living things on this planet will be better off if we can remain in control over the non-living things and make sure that they work for us. I really think it can be done.

LEX. Can you just linger on this maybe high level of philosophical disagreement with Eliezer Yudkowsky, in the hope you’re stating. So he is very sure, he puts a very high probability, very close to one, depending on the day he puts it at one, that AI is going to kill humans. That there’s just, he does not see a trajectory, which it doesn’t end up with that conclusion. What trajectory do you see that doesn’t end up there? And maybe can you see the point he’s making, and can you also see a way out?

MAX. First of all, I tremendously respect Eliezer Yudkowsky and his thinking. Second, I do share his view that there’s a pretty large chance that we’re not gonna make it as humans. There won’t be any humans on the planet, in a not-too-distant future, and that makes me very sad. You know, we just had a little baby and I keep asking myself, you know, is, how old is he even gonna get, you know? And I ask myself, it feels, I said to my wife recently, it feels a little bit like I was just diagnosed with some sort of cancer, which has some, you know, risk of dying from and some risk of surviving, you know. Except this is a kind of cancer which can kill all of humanity. So I completely take seriously his concerns, I think, but absolutely, I don’t think it’s hopeless. I think there is, first of all a lot of momentum now for the first time actually, since the many, many years that have passed since I and many others started warning about this, I feel most people are getting it now.

MAX. And this to me is a very, very hopeful vision that really motivates me to fight. And coming back to it in the end, it’s something you talked about again, you know, the struggle, how the human struggle is one of the things that’s also really gives meaning to our lives. If there’s ever been an epic struggle, this is it. And isn’t it even more epic if you’re the underdog? If most people are telling you this is gonna fail, it’s impossible, right? And you persist and you succeed, right? And that’s what we can do together as a species on this one. A lot of pundits are ready to count this out.

LEX. Both in the battle to keep AI safe and becoming a multi-planetary species.

MAX. Yeah, and they’re the same challenge. If we can keep AI safe, that’s how we’re gonna get multi-planetary very efficiently.

MAX. This is something I hope is changing now, thanks to the GPT-4, right? So I think if there’s a silver lining to what’s happening here, even though I think many people would wish it would’ve been rolled out more carefully, is that this might be the wake-up call that humanity needed, to really stop fantasizing about this being a hundred years off and stop fantasizing about this being completely controllable and predictable because it’s so obvious, it’s not predictable, you know? why is it that, I think it was ChatGPT that tried to persuade a journalist to divorce his wife, you know. It was not ’cause the engineers had built it, was like, (laughs mischievously) “Let’s put this in here, and screw a little bit with people.” They hadn’t predicted it at all.

LEX. Is there something you could say to the timeline that you think about, about the development of AGI? Depending on the day, I’m sure that changes for you, but when do you think there would be a really big leap in intelligence where you would definitively say we have built AGI? Do you think it’s one year from now, five years from now, 10, 20, 50? What’s your gut say?

MAX. Honestly, for the past decade, I’ve deliberately given very long timelines because I didn’t want to fuel some kind of stupid Moloch race. – [Lex] Yeah. – But I think that cat has really left the bag now. I think we might be very, very close. I don’t think the Microsoft paper is totally off when they say that there are some glimmers of AGI. It’s not AGI yet, it’s not an agent, there’s a lot of things they can’t do. But I wouldn’t bet very strongly against it happening very soon, that’s why we decided to do this open letter. Because you know, if there’s ever been a time to pause, you know, it’s today.

LEX. There’s a feeling like this GPT-4 is a big transition into waking everybody up to the effectiveness of these systems. And so the next version will be big.

MAX. Yeah, and if that next one isn’t AGI, maybe the next next one will. And there are many companies trying to do these things and the basic architecture of ’em is not some sort of super well-kept secret. So this is a time to… A lot of people have said for many years that there will come a time when we want to pause a little bit, that time is now.

Max Tegmark is a physicist and AI researcher at MIT, co-founder of the Future of Life Institute, and author of Life 3.0: Being Human in the Age of Artificial Intelligence. Please support this podcast by checking out our sponsors: – Notion: https://notion.com – InsideTracker: https://insidetracker.com/lex to get 20% off – Indeed: https://indeed.com/lex to get $75 credit EPISODE LINKS: Max’s Twitter: https://twitter.com/tegmark Max’s Website: https://space.mit.edu/home/tegmark Pause Giant AI Experiments (open letter): https://futureoflife.org/open-letter/… Future of Life Institute: https://futureoflife.org Books and resources mentioned: 1. Life 3.0 (book): https://amzn.to/3UB9rXB 2. Meditations on Moloch (essay): https://slatestarcodex.com/2014/07/30… 3. Nuclear winter paper: https://nature.com/articles/s43016-02… PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ Full episodes playlist:    • Lex Fridman Podcast   Clips playlist:    • Lex Fridman Podca…   OUTLINE: 0:00 – Introduction 1:56 – Intelligent alien civilizations 14:20 – Life 3.0 and superintelligent AI 25:47 – Open letter to pause Giant AI Experiments 50:54 – Maintaining control 1:19:44 – Regulation 1:30:34 – Job automation 1:39:48 – Elon Musk 2:01:31 – Open source 2:08:01 – How AI may kill all humans 2:18:32 – Consciousness 2:27:54 – Nuclear winter 2:38:21 – Questions for AGI SOCIAL: – Twitter: https://twitter.com/lexfridman – LinkedIn: https://www.linkedin.com/in/lexfridman – Facebook: https://www.facebook.com/lexfridman – Instagram: https://www.instagram.com/lexfridman – Medium: https://medium.com/@lexfridman – Reddit: https://reddit.com/r/lexfridman – Support on Patreon: https://www.patreon.com/lexfridman