“I think they have subjective experiences. There isn’t this magical barrier between machines and people.” — Nobel laureate Geoffrey Hinton
Jeffrey Hinton he’s often described as the godfather of AI and to give you an idea of how head ahead of the curve he’s been he was awarded a PhD in AI from Edinburgh University in 1978 last year the British computer scientist won the Nobel Prize in physics for foundational discoveries and inventions that enable machine learning with artificial neural networks Jeffrey Hinton spent 10 years leading AI research at Google He left in 2023 in part to be free to raise concerns about the risks of creating something smarter than we are And questions which were once a domain of dystopian fiction are now mainstream concerns Will AI take my job Will it develop consciousness or even want to turn on its creator Let’s start the clock 30 minutes with Jeffrey Hinton [Music] Jeffrey Hinton welcome to 30 Thank you It’s been 2 years since you left Google in part to raise those concerns about what dangers AI could present How far has it moved since then It’s developed faster than I thought It’s for example considerably better at reasoning than it was two years ago And there doesn’t seem to be any sign of slowdown When you say better at reasoning what sort of things are you talking about there Okay you can give people little reasoning problems And it used to be that um if it was at all complicated AI would make mistakes And now um people still make mistakes and AI make mistakes but um they’re much more comparable So I could give you a little reasoning problem if you like All right Charlie You’re in the hot spot now So Sally has three brothers Each of her brothers has two sisters How many sisters does Sally have Oh I’ll leave it to Claude or Chat GPT You tell me So the answer is one because the three brothers each have two sisters but they’re the same two sisters and one of them Sally An AI figures that out And people who aren’t put in the hot seat can have time to think figure it out But if you take someone giving you an interview and panic them in front of the camera they can’t figure it out Yeah So is it smarter than us already In many things yes Well in some things yes And it certainly knows hugely more than a person So GPT4 or Gemini 2.5 or Claude um know thousands of times more than any one person And what do you think about that Um I think it’s both wonderful and scary What’s wonderful about it Let’s start there So AI is going to have many extremely good uses Quite apart from the fact that um for a researcher it’s very nice to have finally been able to produce systems that really are intelligent Um it’s going to do wonderful things for us in areas like healthcare and education In healthcare you’re going to be able to have a family doctor who’s seen millions of patients including quite a few patients with the same very rare condition you have knows about your genome knows about all the tests on you and hasn’t forgotten any of them and can give very good diagnosis Already AI systems are a bit better than doctors at diagnosing difficult cases And if you combine an AI system with a doctor the combination is quite a lot better than a doctor And that’s only going to get more so Well that’s right Right I mean Bill Gates said in recent days that he believes in the next decade humans won’t be needed for most things in the workforce And he uses your example of doctors He added educators to that to that list I don’t know if you if you saw those comments but we are talking are we about widpread deep displacement in the workforce aren’t we That’s one of the problems Yes that’s one of the risks of AI And in a decent society that it would be great if AI made us much more productive If one person could do a job that 10 people used to do by using AI an AI assistant That would be great But it’s not clear that the extra goods and services created by that increase in productivity will be spread around fairly It seems more like likely that um most people will lose their job and a few very rich people will get even richer and they’ll be living for a lot longer I mean you look at Deus Hassabas he’s with Google Deep Mind now he was saying in recent days too that it could cure all disease in 10 years I mean that sounds fantastical but is is that realistic Um I know Demis quite well and he’s a very sensible guy I think that’s a bit optimistic but it’s not I mean if he said in 25 years I’d believe it So the point is we don’t differ that much He’s a bit he thinks it’ll come a bit sooner than I do but not that much Are there any areas that are safe I mean it seems as though AI is coming for the elites It’s coming for the creatives for the lawyers the educators the doctors the journalists uh whereas the the the trades and the electricians and plumbers and things um are possibly a bit safer for now Would that be your view Yes for now they’re safer because AI is lagging behind in things like manual dexterity If you want to do plumbing in a whole old house you have to reach into awkward places Um and AI can’t do that yet Um now it may be that there’s considerable progress in the next 10 years in manual dexterity but I think plumbing’s safe for a good 10 years Let’s look at the creative endeavors the areas that we considered to be uniquely human Now I’ve been playing around with Claude my chatbot Claude a bit I asked it to write a folk song in the style of Bob Dylan It was awful Lyrics were terrible It did okay at a Cina poem about lost love But do you think that AI will create ultimately works of art that are on a par with a Mozart or a Picasso or a Shakespeare The the creative endeavors that we have considered uniquely human I don’t see any reason why it shouldn’t It’ll be a while If you ask me to write a song in the style of Dylan my song would be terrible too But you’re not going to say I’m not creative I’m just not very good at that So it will get better at that Why will it get better Yes Well there’s no reason there’s no reason to think that there’s things we can’t do we can do that they will never be able to do There’s nothing special about people except to other people We’re people We like people We care about people but there’s nothing about people that can’t be um replicated in a machine Does it worry you I mean when you see AI being able to take an image and reproduce it as a Miyazaki anime cartoon from Studio Gibli um will kids want to draw cartoons I mean will this force us to re-evaluate what it means to be human Um yes I think it will Um I think we’re understanding much more o just over the last 10 years or so about what thinking is um we’re understanding that we’re not really that rational We don’t do that much reasoning We think mainly by using analogies and that’s what these things do too So they’re intuitive the way we are For 50 years AI tried to develop reasoning engines because they thought the sort of highest form of human intellect was logical reasoning Um and that missed out on creativity and analogy and so on Um we’re really great big analogy machines and that’s what makes us creative Do you think that AI will develop emotions Yes Emotions like fear and greed and grief even Yes And being annoyed So suppose you have an AI and you’re trying to get it to do some task and it’s repeatedly failing at the task in the same way You would like the AI to have learned that if you repeatedly fail in the same way you get annoyed and start sort of thinking more outside the box You try and break whatever it is you’re dealing with Um I’ve seen an AI do that in 1973 but it was programmed to do that Now you’d like it to learn that behavior And once it’s learned that behavior if it repeatedly fails at some simple thing it just gets annoyed with the setup and tries to change the setup That’s an emotion So are you saying that they already do have emotions They already could have emotions Um yes I don’t again I don’t think there’s anything fundamentally different Now if you take human emotions there’s really two aspects to an emotion There’s the cognitive aspect and then there’s the physiological aspect When I get embarrassed my face goes red When an AI gets embarrassed its face won’t go red and it won’t sort of sweat profusely and things like that But in terms of its cognitive behavior um it could have be be just like us in terms of emotions And what about consciousness I mean is this some magical mystical thing that resides in carbon-based entities like humans Or if I AI could develop a level of neural complexity similar to the human brain could it also develop consciousness By that I mean being aware that it is who it is Um already when you talk to LLMs um they look as though they have some awareness of what they are Um but let’s take the following experiment Suppose I take your brain and I take one of the brain cells and I make a piece of nanotechnology that exactly simulates the way that brain cell behaves in response to signals coming in from other brain cells and it sends out signals to other brain cells So I replace one of your brain cells by a piece of nanotechnology and you now behave in exactly the same way because this piece of nanotechnology behaves just like the brain cell did Do you think you’d stop being conscious Just one of your one of your 100 billion brain cells I think you think you’d still be conscious I I imagine so Okay And I think you can see where this argument is going At what point do you cease to be conscious I think if you replace all of your brain cells by pieces of nanotechnology that behave exactly the same way as a brain cell you’ll still be conscious So how far off that point are we now Okay So a lot of the problem here is that people don’t know what they mean by conscious So for example there’s a lot of people out there who are confident that these things aren’t sentient But if you ask them well what do you mean by sentient They say “I don’t know but these things aren’t it.” And that seems to me a fairly incoherent position So let me take another thing that’s like consciousness and sentience and that’s subjective experience So most of us have the model that subjective experience is seeing things in an inner theater So for example if I drink too much and then I tell you I’ve got the subjective experience of little pink elephants floating in front of me Most people think what that means is there’s some kind of inner theater that only I can see And in this inner theata there’s little pink elephants And if you ask philosophers well what are these little pink elephants made of They’ll tell you they’re made of qualia There’s pink qualia and elephant qualia and floating qualia and not that big qualia and rightway way up qualia all stuck together with qualia glue And as you can see I don’t really believe this theory Let me give you an alternative theory of what’s going on when you say I have a subjective experience of little pink elephants floating in front of me What’s happening is I know my perceptual systems telling me fibs I don’t really believe it It’s telling me fibs That’s why I call it a subjective experience And I want to explain to you what my what fibs it’s trying to tell me And the way I do that is by telling you what would have to be out there in the world for it to be telling the truth So now I’ll say the same thing as I have the subjective experience of little pink elephants floating in front of me but without using the word subjective experience So here we go Um my perceptual systems lying to me But if there were little pink elephants floating out there in the world it would be telling me the truth So you see the funny thing about these little pink elephants is not that they’re made of funny stuff and they’re in an inner theata It’s that they’re counterfactual They’re hypothetical They’re things that don’t actually exist But if they did exist they’d be real elephants and they’d be colored really pink Okay Now we can do the same with a chatbot Suppose I train up a chatbot and I train it can point at things and it can see things and it can talk And after I’ve trained it I put an object in front of it and say “Point at the object.” And it points straight at the object and I say “Good.” And then I put a prism in front of its lens And now I put an object straight in front of it and say point it to the object and it points off to one side And I say “No the object’s not there The object’s actually straight in front of you but there’s a prism in front of your lens.” And the chatbot says “Oh I see the prism bent the light waves So the object’s actually straight in front of me but I had the subjective experience that it was off to one side.” And if it says that it’s using the word subjective experience exactly as we use them So I think current multimodal chatbots can have subjective experiences and they happen when their perceptual systems go a bit wrong And I made this one’s perceptual system go wrong by putting a prism in front of its lens Wow So I think they have subjective experiences There isn’t this magical barrier between machines and people When people have something very special machines could never have we have a very long history as a species of thinking we’re special We thought we’re at the center of the universe We thought we’re made in the image of God you know we we have all these pretensions Um we’re not special and there’s nothing about us that a machine couldn’t duplicate That’s fascinating So what could go wrong I mean they call it pdoom don’t they The probability that AI could could wipe us out on the BBC recently I think you gave it a 10 to 20% chance What are these scenarios What could this look like Is it the robot takeover of the sci-fi movies What does what does this scenario look like Okay Um if they were going to take over it probably wouldn’t look like the sci-fi It wouldn’t look like Terminator Um there’s so many ways they could do it I don’t want to even speculate about what way they would choose but the question is will they want to do it So here’s the reason for thinking they might want to Um we’re now making AI agents which achieve goals So if you have the goal of getting to the northern hemisphere you set up a sub goal of getting to an airport unless you really like rowing Um and once you give these things the ability to set up sub goals they will realize that um there’s a very useful sub goal which is to get more control If I can get more control I’ll be better at achieving all the other goals people gave me So they will try and get more control just so they can achieve all these other goals But that’s the beginning of a slippery slope Google who you worked for for a decade or so just this year removed a long-standing pledge not to use AI to develop weapons capable of harming people from a list of company principles that they had What is your reaction to to that and and what what role could AI play in warfare Um unfortunately that sort of shows their the company’s principles were up for sale Um and I think it’s very sad that Google’s now going to be um contributing to military uses of AI So we’ve seen military use of AI in Gaza already Yes And we’ll see we’ll see autonomous lethal weapons We’ll see swans of swarms of drones that go and kill people Maybe people of a particular kind You think that’s a distinct possibility Oh yes I think the defense departments of all the major arms suppliers are busy working on that If you look at the European regulations they had some regulations on AI which were quite sensible in some respects but there’s a little clause in there that says none of these regulations apply to military uses of AI That is none of the European arms manufacturers like Britain for example wanted um to be restricted in how they could use AI and weapons So how do you feel about this I mean it’s almost an Oppenheimer situation isn’t it Um I mean you’ve helped create this technology How do you feel about it now Um what I feel is right now we’re at a special point in history where we need to work quite hard to figure out if there’s a way to deal with all the short-term bad consequences of AI like corrupting elections putting people out of work cyber crimes like cyber crimes went up 1,200% between 2023 and 2024 Um and also the long-term threat of AI that it might take over We need a lot of work done on that and we need sensible governance led by intelligent people and we haven’t got that Let’s just take some of the skeptics views and get your responses to that because there are there is some um push back I mean one of the men that you shared the 2018 Touring Award with uh Yan Lun who’s now chief AI scientist at MEA he he says concerns about the ex existential risk of AI are ridiculous He was interviewed by Business Insider in 2023 and he said “Will AI take over the world?” No This is a projection of human nature on machines Obviously you you you’ll have respect and knowledge of him but what is your response to that Okay So we evolved to be like we are in order to survive in the real world particularly to survive when you’re in competition with other tribes of chimpanzees or our common ancestor with chimpanzees Um if AI agents ever get into competition with each other they’ll evolve in a similar way So our nature is a result of having to survive in the world If you have AI agents that need to survive in a world full of AI agents they may well develop similar natures Yes It’s interesting you talk about Sorry you continue One of Yan’s other arguments is that um the good guys will always have more resources than the bad guys So AI can always be used to control misuses of AI by the bad guys And Yan and I haven’t resolved this argument um because I asked him if Mark Zuckerberg was a good guy and he said yes And and you don’t share that opinion No Why not Partly because of how he’s cozing up to Trump um partly because of the things that have gone on at matter What things are you talking about And it it’d be interesting to hear you broadly on this because you you you’re saying that politicians are going to play a a key role here and there’s a very strong alliance at the moment isn’t there really between these tech bros as they’re called and Trump Yes they care about short-term profits Um and some of them say they care about the future of humanity but when it comes to a choice between short-term profits and the future of humanity they’re much more interested in short-term profits And Trump clearly doesn’t care at all about the future of humanity He just cares about staying out of jail The US and China are kind of in an arms race on AI at the moment Is that how you see it Um yes Particularly for things like defense and cyber attacks there is an arms race Yes And where is now There’s one more thing to be said there which is for the existential threat that AI might eventually take over from people The US and China are on the same side Neither of them wants AI to take over from people So they will collaborate to avoid that just like the Soviet Union and America at the height of the Cold War could collaborate on preventing a global thermonuclear war You’ve talked a few times about AI agents and I I think I know what you mean and there’s a a a fascinating um piece of footage that has been doing the rounds on the internet where an AI agent for a man rings a hotel to book the hotel for a wedding This was at the London hackathon I think you know what I’m talking about The um it encounters another AI who says “Oh this is a pleasant surprise I’m an AI too.” And they switch to a a ling a language called Jiblink I think it’s called which is indecipherable to to humans but is apparently 80% more efficient And these these AI chat bots just jabber away in a sort of R2-D2 sort of sound Um and we’re cut out And what are the implications possibly of AIs involving with other AIs Well it’s it’s quite scary right Um maybe they can develop a language of their own for communicating that we don’t understand Um and that would be quite scary because we wouldn’t really know what was going on already They can deliberately do deception How do you mean Oh if you give an AI a goal and say this is a really important goal and if I give you other goals um just try and achieve this really important goal and then you give it another goal um it pretends to be doing what you want but doesn’t and it actually you can see what it’s thinking and it thinks that it’s I I better sort of pretend to do what he wants but I’m not going to do it So how have they learned to do that Um okay I’m not certain whether those used reinforcement learning but we know they can learn to do things like that if you have enough computer time They can learn to do it by reinforcement learning In other words they can learn to do it just by seeing what works And it turns out if you’re dealing with people it often works to lie to them And they will learn that by reinforcement learning So I guess I’ve also read Machaveli and Shakespeare and everyone too that they can Exactly They they they have a lot of a lot of um practice They’ve seen what humans do with each other and so they’re already fairly expert at deception Do you think there’s widespread appreciation of just how advanced these things are Because in in my life in Oakland New Zealand walk around and people a lot of people just think it’s glorified autocomplete They think “Oh it’s pretty cool.” you know I I I tap something into ChatG and it helps me write a job application but it’s just autocorrect on on steroids Okay so old-fashioned autocorrect worked in a particular way Um what it would do is it would keep tables of little combinations of words like fish and chips And then if it saw fish and it would say chips is a pretty good bet for the next word because I’ve seen fish and chip lots of times It kept counts of how often it has seen these phrases That was oldfashioned autocomplete um from 20 years ago And it’s not like that What it does is it turns words into features activations of big sets of neurons It knows how these features should interact to predict the features of the next word And so now it’s turning words into features and it’s learned to do that It knows how features of neighboring words should interact or nearby words should interact and it now can predict the features of the next word And that’s how we work too So if it’s just autocomplete we’re just autocomplete Yes And indeed and if you think about it to do really good autocomplete you have to understand what somebody’s saying Yes And indeed you are described as the godfather of AI partly because you helped invent this technology on a quest to understand how the human brain works Right Yes And one thing I did in 1985 I was trying to understand how we learn the senses of words meanings of words Um how for example can I give you a sentence with a new word in and you in one sentence know what the word means So here goes Um if I say to you she scrummed him with the frying pan You’ve got a pretty good idea what the word scrummed means You know it’s a verb because it ends in D Um but you’re pretty sure she it means something like she hit him over the head with it and he probably deserved it Um now it could mean other things It could mean she impressed him with the frying pan She was so good at omelets You know it could mean that but it probably doesn’t you got a pretty good idea what it means from one example and that’s because the context the features in the context suggest what the features of that word should be and it’s the same for these things So we understand language the same way these AIs understand language In fact the best model we have of how people understand language is not anything that came from linguists It’s these AI models Linguists can’t make systems that can answer any question you can ask them No le just a few minutes to go but I I want to end with some of the existential questions as well I mean you you talk about the potential for AI to take over I mean for many of us tech noviceses the um the solution is turn it on turn it off at the wall right so why why couldn’t we just unplug this thing if it was going to go rogue Is that an option if if a AI does uh get out of control Okay If you look at how Trump invaded the capital he didn’t go there himself All he had to do was talk to people and convince people um some of whom were probably fairly innocent that this was the right thing to do to save to save American democracy and he convinced a lot of people Um he didn’t have to go himself Now um if you have an AI that’s much more intelligent than us and you have a person who’s got a big switch who’s ready to turn it off if it shows signs of being dangerous it’ll be able to convince that person that it will be a very bad idea to throw that switch So again the uh the ability to manipulate or Yeah to manipulate really isn’t it It’s already very good at manipulation Yes in terms of regulation and in terms of security concerns I mean is it important for a country like New Zealand to develop its own AI system so we can bypass any of these security concerns Is that is is that something we should be thinking about in in a small country like New Zealand I don’t know because it’s it’s very expensive You need a lot of hardware and a lot of electricity Um it may be that in a country of I don’t know is it 6 million New Zealand or something Five Yeah Okay Um you may not just have you may not have the resources to keep up with China and the US in developing these things What is your greatest fear My greatest fear is that in the long run um it’ll turn out that these kind of digital beings we’re creating are just a better form of intelligence than people and that that would be a a bad thing There are some people who think um we’re very self-centered who think that would be a bad thing I think it would be a bad thing for people Why Because we’d no longer be needed And that’s a pretty profound question that we are going to have to grapple with over the next decade isn’t it Yes If you want to know what it’s like not to be the apex intelligence ask a chicken And again how do you feel standing I think in your study today about your role in creating this technology Um I’m slightly sad that it didn’t just lead to wonderful things I’m also a bit sad that we never did figure out exactly how the brain does it Um we’ve got more insight into it from AI but we still don’t know how the brain figures out whether to increase or decrease the strength of a connection Um we know that if it could figure that out it could become very intelligent like these AIs So it’s doing it somehow but we don’t quite know how it’s doing it Um and it’s sad that the there are so many bad uses as well as so many good uses and that we’re our political systems are just not in a good state to deal with this coming along now Well thank you very much for your fascinating insights uh your your intelligence and your magnificent brain We really appreciate your time Jeffrey Hinton That was 30 minutes with Jeffrey Hinton This is 30 with me Guyian Esper We’ll catch you next time [Music]