“We need to prevent AI from becoming completely unfathomable. That if we give it a lot of power to make decisions about human beings, about human society, we need the ability to understand it and to regulate it. This is the crucial condition and there is a very big difference between a situation when 1% of you Humanity understand something and 0% – that’s a very big difference. […] I’m not sure what kind, what money would look like in 20 years or 30 years, but I hope it will be a currency of greater trust and not a currency of distrust.”
TRANSCRIPT. you all uh great to see you here uh we have discussed at this Summit uh where AI is going uh where Financial firms and central banks are planning to hope to do with AI uh but also how they’re planning to manage risks that they can see stemming from AI uh and I hope we can have a discussion around sort of zooming out from the world of of um Ai and and the financial world and and and think about more opportunities and threats for our broader Society uh but if I may I I’ve heard you say that um you don’t see any serious attempts to regulate uh the financial system as we see it in the world to actually be at a level where humans can understand it and given that starting point uh and given uh the uh the issues around AI it’s not looking like the financial system is going to get any easier to understand um so um if we think that this is going to play out how would you think um that regulators and supervisors should should think about um uh AI in the context of the already complicated financial industry we’re operating in well that’s a very important question and I think what we should understand is that people need to understand Finance at the present moment I think that maybe 1% of the population really understands how the Financial system works what happens if this number goes down to zero no human being no president no prime minister no nobody in the Central Bank understand Finance anymore because AI has made it too complicated uh this could lead to a political and social crisis of a Kind we’ve never encountered before it could lead to power shifting from humans to algorithms to Al forms of intelligence and maybe I’ll start by saying what Finance is what does the financial sector do you know Farmers grow food that we eat H uh workers produce the textiles that we wear what does the financial system do it creates trust this is the product ultimately the product of the financial system is trust money for instance is a mechanis ISM to create instant trust between millions of strangers I go into a shop I meet a stranger that I never met in my life I give them this piece of paper and they give me food in exchange and bonds and uh shares and so forth people have invented so many different Financial devices over history but what they really do is create trust between millions of of people so they can share their resources their knowledge and work together towards common goals now until today the only entities that could understand Financial devices like money or bonds or shares and could create could invent new Financial devices with human beings hes did not understand money we bought and sold horses for money but the horses didn’t understand money which is why they couldn’t for instance unite a million horses together now ai is the first tool that we created that can actually understand Finance better than us because you know Finance in the end it’s the easiest thing for an AI to to really understand it’s only data it’s only information to wash dishes AI needs to do something in the in the real world in the physical world that’s difficult but to understand Finance it’s only information in information out now the danger is that AI would be able on the one hand in in the hands of Bad actors AI could destroy trust between people and on the other hand the main trust will be between different AI systems so just imagine a scenario when most of the financial activity in the world is being done by AI humans have lost the ability to understand most of the financial system it’s too complicated for our brains and there is a financial crisis and no un politician is able to understand what is happening we have to rely on the AI to tell us what is the crisis and what are the potential Solutions the political implications are dangerous on on on different levels but most clearly it’s really giving control of a very important part of the world and a very important part of politics to a nonhuman intelligence that we cannot regulate and supervise so I I take from this that this is a very sobering message so let’s let’s unpack that a bit further um um you you can say that okay only 1% perhaps understand the financial system I would claim that uh most most of us happily step into an era plane having very vague understanding of how that works but we trust that people who don’t even know each other we certainly don’t know them uh take us from A to B without the plane crashing uh but it sounds from what I hear you saying that this last percent if that goes down to zero then we are all in in deep trouble uh and and one way to create a catchphrase from what you’re saying is uh no further AI without further trust um and I would say that um we might be in a lucky situation that we are at the early phases of exploring AI or at least at least for the moment assume that uh you have warned that if we’re not careful the decis makers in the future might sort of be reduced to puppets and the real decision makers are just algorithms but um if we don’t want to get on that path or I mean the history is not sort of or if the future is not sort of predetermined we can actually decide where we want to go what do you think we need to do now with the with the tools we have at hand to make sure that we don’t end up in that situation neither in the financial system and and certainly not when it comes to decision making as a whole in our in our countries uh we need to prevent AI from becoming completely unfathomable that if we give it a lot of power to make decisions about human beings about human society uh we need the ability to understand it and to regulate it uh this is this is the crucial condition and there is a very big difference between a situation when 1% of you Humanity understand something and 0% that’s a very big difference and then your example with the airplane is perfect I mean yes this is why we have experts We Trust experts in many different fields with airplanes with medicine it’s the same with Finance we don’t need everybody to understand everything this is this is really impossible but if you think about for instance the the one of the last big financial crisis the one of 20072 2008 it started with a failure of Regulation over new Financial tools new Financial devices the cdos that some very very bright people invented but almost nobody understood and that therefore there was an oversight of Regulation now imagine a situation maybe in 10 years when AIS create a new Financial device which is orders of or orders of magnitude more complic at than cdos you need to go of trillions of of of data data points to to to understand this this financial device we can’t do that but the AIS can for a couple of years it works perfectly well it generates a lot of trust it generates economic growth profits and then something goes wrong and nobody knows why and again we can ask the AI to tell us what has happened and what to do about it but uh this is a very dangerous situation and also the the even if it gives us good advice it it it it raises a question about whether the public will continue to trust the political system like you vote for politicians who are supposed to make these decisions but they are no longer able to understand what is happening we already are in a situ sitation when the public is losing trust in experts in politicians in Democratic institutions um this could really exacerbate this crisis of trust between the general public and the supposed experts and politicians uh that that hopefully understand what is happening and and regulated so going to your um role your day job as a as a distinguished historian uh I mean I I get that there’s uh risks around artificial intelligence we could possibly the greatest transition that the world has has faced but as an historian you know this is not the first big transition we have seen in the world perhaps the biggest but but certainly Industrial Revolution general purpose Technologies the uh the Advent of modern democracies are were were pretty groundbreaking shakeups a couple of revolutions thrown in in a Wars pretty serious shakeups um uh and and um so my I guess my question to you is that um what what can we learn from the past here to try to avoid having to land in that sort of situation that you’re warning from as in historian um all good examples from history where people actually manag to avoid getting things to wrong and that we can take inspiration from yeah I think humans are extremely adaptable and you know every everything we invent has both negative and positive potential you invent a knife you can use it to murder somebody or to save their life in surgery or to cut salad so there is always both of these potentials and uh over time humans learn the dangers and they they understand how to employ the technology of the new structure in in in the best way the problem is the transition period the problem is the path even if in the end we get it right on the way there we often have to go through dangerous and failed experiments so let’s take the Industrial Revolution as an example that when machines started to transform the economy and Society in the 18th century the 19th century there are a lot of Doomsday prophecies about what the machines will do and looking back from 2024 we can say that in the end we got it more or less right we learned how to use steam engines and electricity and trains in order to build more prosperous and even more people peaceful societies than in the 18th century but the the big problem was was the failed experiment on the way uh modern imperialism for instance was a very big failed experiment in how to build industrial societies there are a lot of people in the 19th century that argued that the only way to build an effective industrial society was to build an Empire because unlike previous agricultural societies which were more or less self- sustaining an industrial society needed a lot of raw materials and Global markets and the only way to do that was to build an Empire so imperialism with the the millions of victims and the Terrible Things it did was really a failed experiment in how to build an industrial society communism was another failed experiment in how to build an industrial society and that a lot of people in the 1920s 1930s even later thought that the best way to build an industrial society was what the Soviets were doing and it took decades and tens of millions of victims to realize that no this is not the best way to use electricity and steam engines and trains Nazism was another failed experiment in building an industrial society now if we had to go through these kinds of failed experiments with imperialism and totalitarianism and world wars in order to learn how to make the best use of electricity and the internal combustion engine and trains and things like that what will happen in the 21st century when we have to learn how to use far more powerful technology now ai is again potentially orders of magnitude more powerful than anything we saw in the industrial revolution because AI is the First Tool in history that can make decisions by itself and can create new ideas by itself steam engines could not decide how to use themselves we decided what to do with them even atom bombs could not create new ideas but AI can make decisions by itself this is the very definition of an AI it’s a a machine that can learn things we don’t know and can make decisions independently and can create completely new ideas that no human being have ever thought about learning how to use these tools in a good way is likely to be more complicated than learning how to use steam engines now we can say that in the end it will be okay we’ll get it right as a historian I’m concerned about the way not about the end result if we have to go through another cycle of Empires and totalitarian regimes and world wars based on AI uh this is very very bad news well it sounds like um to see a silver lining in this we will get it right after we’ve tried everything else imperialism communism Nazism and maybe to give a further positive spin is that we and our children grandchild Etc will have knowledge that they didn’t have back then yeah so uh let’s hope that we could sort of fast forward through that period or avoid it all together uh and I think one of the ways to do that is to be a be mindful of uh uh the risks technology is fantastic but used in the wrong way it can be uh very dangerous um I I think I have some people in the audience there might disagree that it’s inevitable that AI will take over decision making but it will certainly influence decision making we have discussed um whether uh monetary policy decisions is possible to automate the answer is yes but as human beings we always want to have um someone accountable or someone to blame uh and that’s a deep human uh um feature in us that we ultimately we want to interact with one another rather than than a machine machine is helpful but you we we are still Still Human enough to to want to uh interact with one another at the end of the day I I have I I think I know the answer to your question now given that um your view about what the financial crisis um the upbringing of the financial crisis some 15 years ago but still I I I’ll have a shot at this so the mission of the bank for international settlements is to support central banks uh through intern National Corporation to pursue monetary and financial stability I think personally the beauty of central banks is that they are not as dependent on um short-term political decision making as many other parts of the public sector so central banks if they operate well and has a wellworking mandate they can actually take the longer perspective um and um do you think that there could be a role for central banks in sort of Drive International coroporation um where AI is used with care uh that the Central Banking Community can sort of show good practice if you will uh and and sure there would be others who has another different agenda uh but that is something that um well we would have to leave to the law enforcement agencies and ultimately military defense but do you think there is a there is some sort of way out of your rather dire forecasts yeah absolutely again I’m if it was doomed there was no point having this conversation that’s reassuring thank you the history is never deterministic we always have options the same technology can always be used in different ways again looking back to the 20th century you look at industrial technology it it was used to create totalitarian regimes but also liberal democracies you look at North Korea and South Korea today they have access to exactly the same technology using it in a very different way now um the the how to make sure that we are using technology wisely I wouldn’t trust some technological Gadget to do it for us we need to have humans in the loop but I wouldn’t trust any individual Geniuses or charismatic charismatic leaders either um the solutions that we again and again see in history is institutions I know that people often find institutions again complicated and and bureaucratic and and but this is the best thing that humans have managed to come up with and we need to build good institutions this is the only thing that can really protect us from the most dangerous potential of of AI and also other Technologies I would say that two things that are very important for a a a bank when coming to to think about the regulation of AI is first of all to make sure that the AI remains accountable and and understandable if we reach a point when we can no longer understand it then at least in Democratic Society is um this is really the end of democracy if if a democratic public can no longer understand and hold someone accountable for financial decisions this is a very large part of what politics is about is about Finance is is about the the the economy and um we of course will need to rely also on technology to help us understand AI but we’ll need to keep humans in the loop and for this we need to build good human institutions the other point is to keep the focus on the issue of trust and going back to what we spoke in the beginning uh money is really trust money is not made of gold or silver or paper or anything money is trust between people this is the goal of the financial system so when you come to regulate the financial system the key question is is how do we make it easier to create more trust between more people if we achieve this then the goal is achieved you said we should keep humans in the loop I think uh we should be more ambitious than that uh and and let me before I open the floor for questions from the audience just point out one thing that is I think strong characteristics humankind which is that um we have dreams um and then some things that we do are not entirely rational and some things can be outright a bit crazy or uh be revealed as a pure accident um let me take some examples um the idea to put a man on the moon uh wasn’t entirely rational would probably not have survived today’s budget committees but it worked in the 60s thank God for that um uh accidental uh discovery was the discovery of penicillin or Alexander Fleming was a very good marketer uh but certainly a very inspiring uh um example I don’t think personally that AI could replicate these sort of events that are not entirely rational or sort of happen by by accident so from where I see it um uh humans will still play an important role to to create the sort of quantum leaps that these two examples uh give uh or do you think that AI will take over those as well I think that a lot of people say that AI will never be creative but it is already very creative uh you look at examples like the game of Go which was one of the the first time that really the world woke up to to the age of AI uh go is a board game a bit like chess but much much much more complicated people have been playing go for 3,000 years in East Asia it has been considered for thousands of years one of the most important out forms that every cultured person should know intire schools of thought about how to play go and different strategies then Alpha go came along and within a few weeks or months revolutionized the game it’s started playing go in a way that no humans ever thought of playing for 3,000 years what turned what what what what became clear is because we are organic beings that evolved by Evolution our imagination and the way we think is limited to let’s say a certain area within the landscape of playing go and for 3,000 years we only exploded this limited landscape this limited area AI is not organic it doesn’t think like a human or a chimpanzee or a dog it makes decisions in a completely different way and within a very short time it started exploring completely new or alen areas within this landscape of go what’s true of go is is true of more and more areas in science in art you know AI is now composing music so people say yes but it only learns from previous human Symphonies and and poems and songs that’s true for now but this is true of us as well a human composer also relies on previous experience or previous exemplars but already now ai is beginning to explore new areas in the landscape of music and maybe most important to understand is that AI is just a tiny baby the AI of today is just the the you know if you thinking again in evolutionary terms it’s like we are 10 years from the beginning of organic evolution on Earth if you go back 4 billion years ago go to the beginning of organic life so think that you are 10 years from the emergence of the first life forms and can you even Envision dinosaurs or human beings now ai is is is just beginning and but it’s its evolution is much much faster than organic evolution it is millions of times faster so if the AI of today are like amibas we might encounter the AI tyranosaurus Rex within a decade or two and we have no idea what it will be capable of doing again all these discussions of AI 10 years ago nobody except a very small community of experts was talking about AI so think about the previous 10 years and try to imagine where we will be in 10 or 20 years and uh in terms of creativity I have no doubt given what we already know that AI is capable of maybe a different kind of creativity but precisely because of that it’s really going to be a game Cher in in so many fields because you know the the the initials AI people think it stands for out arcial intelligence but there is nothing artificial about the new forms of of of AI they are developing on their own it’s better to think about AI as standing for alien intelligence because it really thinks and creates in an alien way a different way from us and this is even true of Finance again I think about financial device as a kind of out for the first person who came up with the idea of a bond or of a currency or of Bitcoin this is a sort of art it’s very creative very imaginative so I think we’ll see AI creativity also in this field coming coming up with new types of financial devices that no human being would ever imagine creating I would say that uh those financial um products that you pointed out to they came up because they satisfied someone’s need and I think that is something that is hugely important uh among human beings that we are very good at figuring out a what needs do we have do we want to satisfy sometimes we don’t know that we had needs that we wanted to satisfy with Steve Jobs with his iPhone they explored uh but uh I I’m more optimistic than you uh and I’m going to stick to those institutions actually uh because I think I agree that that those are our best way to organize and coordinate uh sort of AI development in a in a responsible way not everybody will join that Journey uh but if enough people do it enough institutions are on that journey I think we I think I still think we have a good shot I I certainly like being an optimist uh more than anything else but uh let’s open up the floor for questions from the audience and please state your name and and I think we have a new record in the name number of so let’s start with the gentleman up to the left side and please state your name first and then your question to M Professor Harari thank you Cecilia um my name is hien um thank you very much um to Professor Hari as well it’s been really privileged listening to you so my question is really goes in the same ve of of a recent discussion is about human versus AI um the human is more intelligent ingenious form right I mean it’s organic form we know how to sustain and we sustain over last years you know the climate crisis um you know so where we are today and if you look at the future um we cannot look at the AI in isolation right there are other external factors to play and in particular the climate right the sustainability aspect um AI is feed on electricity electricity then feed on these sort of Natural Resources which are dep depleting and these kind of things so how do you sort of see that you know AI is sort of surviving or AI sort of overcoming those challenges which human have and human still have the ability in sort of to go through those kind of another um round of challenges you know um so just wanted to get your perspective on this that energy uh consumption is going to hold back on the AI development it it well energy consumption as as we uh moves low right as we actually increase more capacities um there will be more consumption right and if we actually start feeding more energies and electricity I think it’s going to have the sort of downside effect on the environment and the natural resources all of these things so somewhere actually it’s going to come to the Hal right um now human can survive without electricity but AI would not right I that’s a that’s a cude example probably your question again is well I think understood the question okay and I on this I may have something more positive to say uh and I’m not against AI I don’t think we can just stop developing it and I think it has a lot of positive potential one of the positive potentials is to help us deal with the climate crisis and with the energy crisis I don’t think that it’s uh given the the current political and economic situation around the world uh solving the climate crisis by stopping all economic growth is is impossible and and unwanted because there are billions of people around the world whose living conditions need a significant Improvement and you know if every person in India in Africa in South America enjoys the same living conditions as the average person in Europe or the United States with current technology the ecology will collapse but on the other hand we can’t say to people in India or in Africa no you cannot enjoy what the Europeans have so we need to develop better technology uh more eco-friendly technology and we need to do it fast and this is something that AI when it’s used correctly can certainly help help us with uh the basic uh understanding is that there is no shortage of energy in the universe we just don’t know how to tap it and how to use it in the in the best way in the most eco-friendly way and I think this is one field where AI can be extremely helpful thank you let’s take a question on this side so um why don’t we take one uh over there yeah you get to choose if you you need to hold your hands up otherwise you will lose the chance woman that’s nice so thank you very much um it’s a pleasure and an honor to be talking to you your books are fantastic so uh Gemini happened um Google’s Gemini this was a couple of months ago uh many would argue that there was decision to put the finger on the scale so that the the outcomes that individuals would see when they were using Gemini were much more PC I don’t know if everybody in the room is familiar with this this but you would put in show me pictures of the founders of the United States and you would see an African-American woman um I was in Silicon Valley when this just happened and and I was talking to different people out there like where does this go because in the end these are individuals that are prompting seeking information and getting feedback from algorithms and some of the libertarian points of views that were shared with me are in the end individuals are going to see what basically the AI has been trained that they want to see it’s the human bias that this is going to be almost like a reinforcement Loop and and I’m trying to and and so then the next question is so okay in the United States we have the southeast where we have maybe we see lots of people of a certain race and other places right like where does this go and so I’m I continue to poke at this question and I’m trying to Recon it with what you saying about it’s trusted institutions because I’m having a hard time seeing how this sort of plays out in a way that isn’t what we’re all afraid of you know humans attacking each other yeah I mean you know we had these kinds of of debates and problems throughout history every time a new information technology comes along there is a big discussion about censorship and you know when when the printing press was invented the big best sellers of early modern Europe they were not scientific books hardly anybody read Galileo cicus the big best sellers were uh uh witch hunting manuals how to identify and kill witches very popular also popular were these extremist religious literature the printing press was to some extent responsible for the Scientific Revolution but also for the European Witch Hunt craze which was a modern not a medieval phenomenon and also for the wars of religion so they had a big issue with what kind of things you are allowed to print and we have a big issue with what kind of things AI is allowed to say the difference is that AI has agency unlike printing presses the printing press printed whatever you told it to print AI is learning by itself any attempt to try and kind of in advance determine what these AIS will say down the road is ultimately doomed to failure because they are learning and developing this is a huge issue for democracies it’s even a bigger issue for dictatorships one thing that people often miss is that dictators are terrified of of these AIS even more than democratic governments because think about it what is the most dangerous thing for a dictator the most most dangerous thing for a dictator is a subordinate more powerful than the dictator that the dictator does not know how to control dictators usually control their subordinates their agents by terrifying them but you cannot terrorize and AI think about a chatbot let’s say in the in in in Russia now a human is afraid to say certain things in Russia because he or she could be punished but you cannot send a chatbot to the gulag so how do you make sure that the chatbot doesn’t start to say all kinds of things that it is not supposed to say and um and there is in in essence it’s a new challenge for all kinds of human regimes because it’s the first time in history that regimes either Democratic or or dictatorial need to figure out how to control agents entities with agency which have no emotions which cannot be terrorized or punished or or or whatever so it’s it’s it’s going to be a very interesting right for for all governments around the world how to deal with this issue no doubt uh let’s have one last question on this side uh Vincent you get to choose again or if you can combine your questions maybe you can coordinate there yeah we’ll see but these two are your last my name is fabiio and I make it very short I liked what you mentioned money is trust among people and the goal is to have more trust with more people could you elaborate a bit what would be the best next action to reach that goal and maybe as a side remark is this maybe the chance to overcome the concept of of Nations I think we need at this stage to overcome the concept of Nations gentlemen at the front do you have a related question or something completely I do all right fire aways um because this is because Nations I think in in your terminology on your thought would be uh would be fictions um so so um You On a related note um the space that the human go player can operate in is limited by its evolutionary biology and so on right whereas Nai is not um the fictional constructs that we can come up with as human beings are limited by our our evolutionary biology as well um but an AI is not could there be fictions that could that could be could result in human flourishing which far exceeds that which uh we could come up with on our own and are we morally obliged therefore to give up to our robot overlords and allow them to determine what the fictions should be that we are then pursuing to maximize human flourishing all right the end of Nations and the well sort of end of humans uh you all have a short the short the short answer is no very dangerous to just trust the AIS yes they can create completely new kinds of fictions but fiction again it can be helpful it can be very dangerous nations are a good example yes Nations is something that we created in our imagination they exist only in our Collective imagination um they can be very dangerous if we don’t know how to use them leading to Wars and conflict and so forth but they are also so far one of the best inventions one one of the most best creations of humanity uh because Nations enable us to go beyond the immediate circle of friends and family that we know personally and care about millions of strangers that we never met you know patriotism in its in its best form what patriotism means is that I’m paying taxes so that strangers on the other side of the country will get good health care this is patriotism and people who imagine that without Nations we would be living in some kind of of a kind of Utopia no at present we would just disintegrate into tribal chaos with people caring only about their immediate uh uh friends and family yes we need to find ways for Nations too to cooperate between them but this is the key not to abolish Nations we are not there yet in in terms of what we are capable of historically the question is how do we make them cooperate instead of fighting each other and uh here yes we need new imaginative Creations also in the field of Finance which again should focus on building trust so to take one a famous example if when I look at at Bitcoin as an as as as a as a historian I don’t like it because this is a money built on distrust the central idea of Bitcoin is basically electronic gold that we don’t trust the banks the governments so we don’t want to give them the ability to create as much money as they like so we create this Bitcoin it’s a currency of distrust I do think that the future belongs to electronic money but what we’ve seen over the last centuries is that it’s actually a good idea to give Banks and governments the ability to create more and more money uh in order to build more trust within Society so I’m not sure what kind what money would look like in 20 years or 30 years but I hope it will be a a currency of Greater trust and not a currency of distrust we get it done and it’s also reassuring to hear uh that institutions as a future and countries has a future that will be uh happy dreams for me tonight and uh we’re going to talk to you at some point in the future and hope that we are still here in Flesh and Bones and not in our algorithmic forms thank you very much Professor arari I can say that I have a very happy audience in front of me and we’re giving you a hand for us