A very good analysis from a respected source! (Russell and Miles)
Ten [incorrect and/or absurd] reasons not to care about AI Safety.
- Reason one. We’ll never actually make artificial general intelligence.
- Reason two. Well maybe we will make AGI at some point but it’s far too soon to worry about it now.
- Reason three. Worrying about AI safety is like worrying about overpopulation on Mars.
- Reason four. Well, look, if you’re worried about it being unsafe because the goals are bad, don’t put in bad goals.
- Reason five. Well, we can just not have explicit goals at all.
- Reason six. I don’t think we should worry because we’re not going to end up with just independent AIs out in the world doing things. There’ll be teams with humans and AI systems cooperating with each other.
- Reason seven. But this is science. We can’t control research. We can’t change what people work on.
- Reason eight. You’re a bunch of Luddites. You’re just against AI because you don’t understand it.
- Reason nine. Well, if there’s a problem, no problem we’ll just turn it off.
- Reason ten. You trying to get us all defunded? Isn’t talking about risks kind of bad for business?
Hi. Stuart Russell is an AI researcher who I’ve talked about a few times on this channel already. He’s been advocating for these kinds of safety or alignment ideas to other AI researchers for quite a few years now and apparently the reaction he gets is often something like this. In stage one we say nothing is going to happen. Stage two we say something may be going to happen but we should do nothing about it. In stage three we say that maybe we should do something about it but there’s nothing we can do. In stage four we say maybe there was something we could have done but it’s too late now. So he’s put together a list of some of the responses that people give him. That list is included in a paper, in some of his talks, and in his recent book Human Compatible which is very good by the way. Check that one out. But as far as I know it’s not yet a standalone YouTube video even though it’s perfect YouTube video material. It’s 10 reasons why people who aren’t you are wrong about AI safety. So that’s what this is. 10 reasons people give to not pay attention to AI safety. Now before I start I need to do a sort of double disclaimer. Firstly as I said this is not my list it’s Stuart Russell’s and he gets the credit for it. But secondly Professor Russell and I are not in any way affiliated and I’ve adapted his list and given my own take on it so this video should not be considered a representative of Stuart Russell’s views. Got that? If there’s anything in this video that’s good credit goes to Stuart Russell and if there’s anything that’s bad blame goes to me. Okay without further ado 10 reasons people give to not pay attention to AI safety.
Reason one. We’ll never actually make artificial general intelligence. Reason 1 Will never actually make AI This is apparently a fairly common response even from AI researchers which is very strange when you consider the decades that the field of AI has spent defending attacks from the outside from people like Hubert Dreyfus, who I have a video about, people arguing that human level AI is impossible. AI researchers have always said no of course it’s possible and of course we’re going to do it. And now people are raising safety concerns. Some of them are saying well of course we’re never going to do it. The fact is that human level intelligence, general intelligence, has been a goal and a promise of the field of artificial intelligence from the beginning and it does seem quite weird to say yes we are working towards this as hard as we can but don’t worry we’ll definitely fail. Imagine you find yourself on a bus and the bus driver says oh yeah I am driving towards the edge of that cliff but don’t worry the bus is bound to break down before we get there. Now they’re not necessarily being disingenuous they may actually believe that we’ll never achieve AGI but the thing is eminent scientists saying that something is impossible has never been a very reliable indicator. I mean they certainly say that about a lot of things that really are impossible but they also say it about a lot of things that then go on to happen sometimes quite quickly. For example respected scientists were making public statements to the effect that heavier than air human flight was impossible right up until the Wright brothers made their first flight at Kitty Hawk and in fact I think even slightly after that because the news traveled fairly slowly in those days. Similarly, and this is something I talked about on Computerphile, the great physicist Ernest Rutherford, Nobel Prize winner Lord Rutherford in fact at that time, gave a speech in 1933 in which he implied that it was impossible to harness energy from nuclear reactions. He said anyone who looked for a source of power in the transformation of the atoms was talking moonshine. That speech was published in the times and Leo Szilard read it, went for a walk and while he was on his walk he had the idea for using neutrons to make a self-sustaining nuclear chain reaction. So in that case the time taken from the world’s most eminent scientists claiming that a thing was completely infeasible to someone having the idea that makes it happen was as far as we can tell somewhere around 16 hours. So it’s happened several times that something which the best researchers say can’t be done has been achieved soon after. Do I think AGI is going to be discovered very soon? No. But I can’t completely rule it out either. We don’t know what we don’t know. So it seems pretty clear to me that unless we destroy ourselves some other way first we will sooner or later figure out how to make human level artificial general intelligence.
Reason two. Well maybe we Reason 2 Well maybe we will make AI will make AGI at some point but it’s far too soon to worry about it now. Suppose we detected a huge asteroid on a collision course with earth. It’s one of those mass extinction event type asteroids and it’s going to hit us in say 40 years. How soon would be too soon to start worrying, to start working on solutions? I would say it’s pretty sensible to start worrying immediately. Not panicking of course that’s never useful but at least spending some resources on the problem, gathering more information, putting together a plan and so on. Or suppose SETI got an extraterrestrial message that said hey what’s up humanity we’re a highly advanced alien civilization and we’re on our way to earth. We’ll be there in say 50 years. Again how long should we just sit on that and do nothing at all? How long before it’s sensible to start thinking about how we might handle the situation? I would say the question doesn’t just depend on how long we have but how long we need. What are we going to have to do and how long is that likely to take us? If we need to build some kind of rocket to go and divert the asteroid how long is that going to take and how does that compare with how long we have? The thing is in the case of AGI we really don’t know how long it will take to solve the alignment problem. When you look at it and consider it carefully it appears to be quite a hard problem. It requires technical work that could take a long time but it also requires philosophical work. It seems like it might depend on finding good solutions to some philosophical problems that people have been wrestling with for a very long time. We don’t have a great history of solving difficult philosophical problems very quickly so it seems to me entirely plausible that we’ll need more time to solve this problem than we actually have. And of course we don’t know how long we have either. Probably it’ll be a long time but predicting the future is extremely hard and we can’t rule out shorter timelines. It could be that we’re closer than we think we are and deep learning will just scale up to AGI without many major innovations. Probably not but it could be. And it doesn’t seem impossible that we could have a Rutherford and Szilard type situation. It could be that there’s one weird trick to general intelligence and once someone discovers that full AGI is only a couple of years away. In fact it’s not totally impossible that someone already found it and has been working on it for a while in secret. None of these seem very likely but confidently declaring that it’s definitely too soon to even start working on this is bizarrely overconfident. The lower estimates for how long we have seem a lot lower than the higher estimates for how long we need. The best time to start working on this is not far in the future. It was probably quite a while ago now.
Reason three. This Reason 3 Overpopulation on Mars is similar to reason two. Worrying about AI safety is like worrying about overpopulation on Mars. I don’t think this is a very tight analogy for a few reasons. One is that overpopulation is one of those problems that very definitely cannot sneak up on you. Overpopulation can’t take you by surprise in the way that a new technological development can. But also the safety concerns we’re talking about are not like overpopulation. They’re much more immediate and more basic. It’s not like what if Mars becomes overpopulated. More like we don’t have any very good reason to expect to be able to survive on Mars for even a day. And there are projects currently underway trying to create AGI. So it’s as though the Mars mission project is already underway and one engineer says to another, you know, we’re putting all this work into getting people to Mars, but almost none into what we’re going to do if we actually manage it. I mean, how do we even know that Mars is safe? Well, I think it’s too soon to worry about that. Don’t you think we should wait until we get there? No, no, I don’t think we should. I think it might be much more difficult to work on these kinds of concerns once we’re already on the surface of Mars. I think we should do the safety research ahead of time. What kind of safety research do you want to do? Well, I was thinking it might be good to have some kind of suit that people could wear in case the environment of Mars turns out to be harmful in some way. We don’t know that the surface of Mars is harmful. Could be anything. Well, exactly. We don’t know. So why not take precautions? What we do know doesn’t seem great. I mean, our what to do if we make it to Mars research has never had much funding, so we can’t be sure about this, but our preliminary work seems to suggest that the atmosphere of Mars might not actually be breathable. So we’ve been thinking about things like suits and there’s some early work on something we’re calling an airlock that might turn out to be useful. I don’t see how we could possibly hope to design anything like that when we don’t even know what Mars is going to be like. How are we going to have any chance of designing these safety features properly with so little information? No, we’re just going to go and we’ll figure it out when we get there. Could I maybe stay on Earth? No, everybody’s going together all at the same time. You know that. All of humanity. It’s going to be great. Reason 4 Bad Goals
Reason four. Well, look, if you’re worried about it being unsafe because the goals are bad, don’t put in bad goals. It won’t have human goals like self-preservation if you don’t put them in there. I have a whole video about the concept of instrumental convergence, which I’d recommend checking out. But as a quick summary, there are certain behaviors that we would expect to be exhibited by agents that have a wide range of different goals because those behaviors are a very good way of achieving a very wide range of goals. Self-preservation is a good example. Agents will act as though they have goals like self-preservation even if you don’t explicitly put them in there, because it doesn’t really matter what your goal is. You’re probably not going to be able to achieve that goal if you’re destroyed. So we can expect agents that are able to understand that they can be destroyed to take steps to prevent that, pretty much whatever goals they have. You can’t fetch the coffee if you’re dead.
Reason five. Well, we can just not have Reason 5 Implicit Goals explicit goals at all. I think this confusion comes from the fact that a lot of the time when we’re talking about safety, we’re talking about the problems we might have if the system’s goals aren’t well aligned with ours, or if the goals aren’t specified correctly, and so on. And this can make people think that we’re talking specifically about designs that have explicitly defined goals. But that’s not actually the case. The problems are much more general than that. We’d expect them to occur across a very wide range of possible agent designs. It’s just that the ones that have explicitly defined reward functions or utility functions are much easier to talk about. The systems with implicit goals still probably have these problems, but it’s just much harder to characterize the problems and think about them, and therefore correspondingly much more difficult to actually deal with those problems. So systems with implicit goals are actually less safe, just because it becomes much harder to design them safely. Not having explicit goals doesn’t solve the problem and probably makes it worse. I have some safety concerns about this car, this automobile that we’re inventing. I’m worried about the steering system. Yeah, I just don’t think we’re putting enough thought into designing it to be really reliable and easy to use. Right now it seems like even tiny mistakes by the operator might cause the car to swerve out of control and crash into something. It could be very dangerous. You think the steering system is a cause of safety concerns, do you? Yeah. Well, okay, yeah, we’ll just build a car without one. Problem solved.
Reason six. I don’t think we should worry because we’re not going to end up Reason 6 Human AI Teams with just independent AIs out in the world doing things. There’ll be teams with humans and AI systems cooperating with each other. We just have to have humans involved in the process, working as a team with the AI, and they’ll keep things safe. So yes, a lot of the better approaches to AI safety do involve humans and AI systems working together, but just have AI human teams is sort of like saying nuclear power plant safety isn’t really a concern. We’ll just have some humans running it from a control room in such a way that they always have full control over the rate of the reaction. Like, yes, but that’s not an actual solution. It’s a description of a property that you would want a solution to have. It would be nice to build AGI systems that can work well in a team with humans, but we don’t currently know how to do that. What it comes down to is teamwork is fundamentally about pursuing common goals. So misalignment precludes teamwork. If the AI systems goals aren’t aligned with yours, you can’t collaborate with it. It wants something different from what you want. So human AI teams aren’t a solution to the alignment problem. They actually depend on it being already solved.
Reason seven. But this is science. We can’t Reason 7 Science control research. We can’t change what people work on. Sure we can. Of course we can. We’ve done it loads of times. It ends up not being very noticeable because we generally don’t give that much attention to things that don’t happen. But for example, we basically don’t do human genetic engineering or human cloning. We’ve been able to clone sheep since 1996 and humans are not really different from any other large mammal from the perspective of this kind of work. We could have been doing all kinds of mad science on human genetics for decades now, but we decided not to. There were conferences where agreements were reached. Everyone agreed that they weren’t going to do certain types of research and then we didn’t do them. So agreements within the research community are one way. Another way is international treaties. Like, did you know that the 1980 United Nations Convention on Certain Conventional Weapons has a section titled Protocol on Blinding Laser Weapons? Because of that protocol, robots that deliberately shine lasers in people’s eyes to blind them are against international law. I didn’t know that until after I’d already built one. So it’s not a perfect metaphor, but the point is we don’t see blinding laser weapons deployed on the battlefield today. They’re basically not a thing. And human genetic engineering is also not really a thing because we decided that we didn’t want to do them and so we didn’t do them. And by the way, if we decide that we don’t want to make, for example, lethal autonomous weapon systems, we don’t have to make them either. AI researchers as a community can decide the direction of our research and we should.
Reason eight. You’re a bunch of Luddites. You’re just Reason 8 Luddites against AI because you don’t understand it. So in response to this, Stuart Russell has a list of people who’ve raised basically this concern, which includes Alan Turing, I.J. Good, Norbert Wiener, Marvin Minsky, and Bill Gates. And there’s another name I would add to this list, which I guess Stuart Russell is not allowed to add, which is Stuart Russell. It doesn’t seem reasonable to suggest that these people fear technology because they don’t understand it, to say the least. These are some of the biggest contributors to the technological progress of the last century. And secondly, these people aren’t against AI. The argument for AI safety is not an argument against AI, any more than nuclear physicists or engineers who work on containment or waste disposal are somehow against physics. Arguing for speed limits and seatbelts is not the same as arguing to ban cars. We’re not against AI because we don’t understand it, we’re for safety because we do understand it.
Reason nine. Well, if there’s a problem, Reason 9 No Problem we’ll just turn it off. Right. This one I’ve covered extensively elsewhere, links in the description. But in summary, I think super intelligent agents might see that coming.
Reason ten. Safety Reason 10. Ixnay on the isksray. You trying to get us all defunded? Isn’t talking about risks kind of bad for business? I’d say, firstly, I don’t think that’s actually true. We do need to make some changes, but that’s not a bad thing for AI research. The solution to these safety concerns is not less AI research, but more really just with a slightly different focus. And in fact, I think this is the same kind of mistake made by the nuclear industry in the 50s. They put tremendous effort into reassuring everyone they were safe. They insisted nothing could possibly go wrong. Nuclear energy was going to be completely safe and perfect, clean and too cheap to meter. Basic reactor principles and design make an atomic explosion an impossibility. Arguably, they were so busy reassuring people about safety that they actually didn’t emphasize safety enough internally. That’s how you get a Chernobyl. That’s how you get a Three Mile Island. And that’s how you get a giant public backlash that is tremendously bad for the industry. So I don’t actually think that talking about AI safety too much is bad for business, but it couldn’t possibly be worse than talking about safety too little. So there we are. That’s 10 reasons not to care about AI safety. Some of these I’ve already covered in more detail in other videos. There will be links in the description to those. And some of them might deserve a whole video to themselves in future. What do you think? Are there any you’d like to hear more about? Or have you come across any that I missed? Let me know in the comments. I want to end the video by thanking my wonderful patrons. It’s all of these people here. In this video, I’m especially thanking Francisco Tolmaski, who has been a patron for a year. Thank you so much for your support. I’m trying out a new thing where, as part of the process of making videos, I talk to the researchers who wrote the papers I’m talking about. I’ll record those for reference, but they’re not really right for YouTube because they’re unstructured, unedited conversation. So I think I’m going to start posting those to Patreon. If that sounds like something you might want to watch, consider becoming a patron. Thanks again to those who do, and thank you all for watching. I’ll see you next time. Bye.
Why do some ignore AI Safety? Let’s look at 10 reasons people give (adapted from Stuart Russell’s list).
Related Videos from Me: Why Would AI Want to do Bad Things? Instrumental Convergence: 
• Why Would AI Want to do Bad Things? I… Intelligence and Stupidity: The Orthogonality Thesis: • Intelligence and Stupidity: The Ortho… Predicting AI: RIP Prof. Hubert Dreyfus: • Predicting AI: RIP Prof. Hubert Dreyfus A Response to Steven Pinker on AI: • A Response to Steven Pinker on AI Related Videos from Computerphile: AI Safety: • AI Safety – Computerphile General AI Won’t Want You To Fix its Code: • General AI Won’t Want You To Fix its … AI ‘Stop Button’ Problem: • AI “Stop Button” Problem – Computerphile Provably Beneficial AI – Stuart Russell: • Provably Beneficial AI | Stuart Russell