Learn more:
308 views 22 Jan 2025
As generative AI systems become increasingly powerful, how can researchers ensure their safety? In this talk, Max Tegmark, physicist and prominent AI ethics advocate, explores the principles and frameworks for building “guaranteed-safe” generative AI. Speaking to an audience of AI researchers, Tegmark focuses on the technical, philosophical, and societal challenges of aligning AI systems with human values. In this video: The definition of “guaranteed-safe” generative AI and why it matters How current generative AI models (like GPTs) risk unintended consequences without rigorous safeguards Technical strategies for achieving alignment and minimizing risks, including robustness testing, interpretability, and scalable oversight Philosophical and ethical dilemmas: balancing innovation with safety in open-ended AI systems Policy implications and global cooperation for regulating generative AI responsibly Max Tegmark calls for deeper collaboration between researchers, ethicists, and policymakers to address the unique challenges posed by generative AI. With his characteristic clarity, he outlines actionable strategies for researchers committed to shaping a safer AI future.
okay so our next speaker is Professor Max techmar which is a professor as MIT and his research in uh incompasses cosmology physics and machine learning he serves as the president of the future of Life Institutes advocating for the beneficial use of technology and addressing extension risks associated with AI the professor techark has ordered over 200 Publications and uh many popular science books so let’s welcome uh [Applause] Max thank you it’s a great pleasure and honor to be back here yeah so eight years ago I decided to switch my MIT research group from physics to machine learning and this is just so much fun I’m an optimist and uh I’m going to argue today that we can have a really really inspiring future with guaranteed safe generative AI if we avoid building AGI anytime soon which I’m going to argue is unnecessary undesirable and and and preventable now the connect this is kind of natural sequel to what yosua just told you because he made a very nice case that non agents are safe and can give us great benefits non- agents being things that have no goals I’m going to add further optimism to this and argue that actually in the v diagram there are even more things that we can also make safe which act which are what I call tools that might have goals but they’re controllable so as long as we stay away from the red stuff so I’d like to start with with u a show of hands here who wants us to make AI tools that can help us cure diseases and solve problems that’s a lot of hands all right now who wants AI that just makes us all economically obsolete and and replaces us see one hand over there two hands so pretty lopsided vote there now there are glooms who who claim that we can’t have what almost all of you voted for there without also the second part being disempowered etc etc I think I’m going to argue that that’s that is really a myth we can actually have the cake and eat it and I’ll spend the rest of my talk explaining why why that is the part that almost nobody voted for here AI that just makes us economically Obsolete and and replaces us on the job market and maybe later even altogether is basically AGI right as yosha mentioned smarter than human AI that can outpour perform us at virtually all tasks and as yosha mentioned that includes by definition the task of AI does development so if you have a bunch of robots that are just much smarter than us and can build robot factories and make new robots Etc and soon you have much more robots than you have people uh it’s it’s pretty natural that we might lose control over them we’ we haven’t made just a new technology now like the internet but a new species so why am I so optimistic suppose I told you that or suppose you someone told you that you can’t that you have to have completely unre unregulated biotech with Eugenics and human cloning Etc otherwise it may be gain of function research or you could lose control over otherwise you can’t have any medicines whatsoever you would call BS on that obviously because you know there’s the third way if you have safety standards you create an incentive for companies and academic to work really hard to figure out how to meet those standards and produce all the wonderful safe medications that we have today and in the same way if someone tells you that you have to have completely uncontrollable AGI otherwise you will’ll never be able to have ai that cures cancer folds proteins or makes awesome self-driving cars or whatever you should call BS on that for the same reason because obviously if we have safety standards then we’re creating great incentives for industry and academics to innovate and meet those standards and give us all these great tools but you might say AGI is really just science fiction and and decades away so why don’t you just shut up and get off the stage and come back in 30 years Max well we you this used to be a very popular position to hold because frankly AI used to suck and this next one is even that was very this next this is particularly embarrassing to me to show this video cu the next one is actually the MIT robot coming up here but you know welcome to 2024 AI doesn’t really suck anymore it’s remarkable how much progress we’ve had now we have robots not only they can dance but they can do all sorts of other things even fold your laundry so and in terms of generative AI the theme of this session you know it went in one year from producing this to producing this with exactly the same prompt and uh as recently as 5 years ago most of my AI research colleagues predicted that something that could Master language and knowledge at the level of Claude or GPT 3.5 or or GPT 40 or01 you know was was decades away maybe 2050 maybe 2040 that was obviously wrong because we have it now and when yosua argued last year that um AI can now pass the touring test I I agree with that and the forward momentum looking ahead is just massive if you add up the entire expenditure on the Manhattan Project from 20 from 1942 to 1946 and adjust for inflation it’s less than a quarter of what was spent just last year on AI and this massive forward momentum of course has really shifted the timelines in many people’s mind you can see it on prediction markets where the time remaining to AGI has dropped from decades away to two or three years away and people like Dario amod from anthropic put a number on and he thinks it’s two years Sam Alman has this to say this is the first time ever where I felt like we actually know what to do like I think from here to building an AGI will still take a huge amount of work there are some known unknowns but I think we basically know what to go what to go do I disagree with with Sam on on many important things but I agree with them here I think there’s no strong argument why we should be super sure that that this is far away and I think uh we have to stop taking for granted that AGI is a long-term issue or someone might make fun of us for being dinosaurs you know stuck in in in 2021 but you might say AI is necessary because we’ve been in urps all week hearing people talk about all these exciting things and we’re going to lose out on all that stuff if we can’t make AGI right no this is totally a myth that we need AGI to get ai’s upside I’m going to argue that it’s actually unnecessary for that we can get basically all of the exciting benefits I’ve heard about all we car in NS without AGI in the near term for example we can save about a million lives on the world’s roads by eliminating traffic accidents with autonomous vehicles using tool aai without AGI tool aai can save even more lives in hospitals without AGI Tuli can diagnos know it’s prostate cancer lung cancer eye diseases and countless other things democratizing access to Fantastic Healthcare without AGI Tuli can help us fold proteins and and revolutionize drug development and Drug Discovery and even get you the Nobel Prize without AGI Tuli can um help us with pandemics Tuli can help save energy tool AI can help make education more accessible and personalized tool AI can transform basically every sector of the economy to produce goods and services more efficiently without AGI Tuli can as yosua mentioned also help us attain the sustainable development goals faster this is from a nature paper I was involved with recently and so in summary it’s really a myth that we need to rush to AGI as fast as possible otherwise somehow all the excitement here at neurs is going to go away that’s just not true but you might say AGI is controllable anyway so why don’t Max why don’t you just go sit down and relax there’s nothing to worry about here I want to just add a little bit to yoshua’s argument that no it’s not once these artificial intelligences get smarter than we are then they will take control they’ll make us irrelevant um and that’s quite worrying and nobody knows how to prevent that for sure so you notice very many leading AI researchers are saying this and it’s quite striking that the main people who are not it’s very hard to find prominent AI researchers saying it who don’t have some sort of financial conflict of interest with with companies and it’s an old idea Alan Turing said this what you heard Jeff Hinton say there you know and what you heard yosu say Alan Turing said it in 1951 already expect the machines to take control it’s it’s important to remember that like when touring says things like this he’s not thinking of if you just think of AGI as a new technology like the steam engine or the printing press or the internet that feels really like hyperbole right but he’s clearly thinking about it as a new species instead as I mentioned in the beginning if you have systems that are can do everything better than us including building copies of themselves in robot factories and stuff that is the definition of a species and then it’s very natural that the smarter species controls why do we humans control Tigers because we’re smarter right and it has nothing to do with some weird speculation about the AI turning evil turning conscious the concern is simply turning very competent and having goals that aren’t aligned with ours as yosua mentioned and U if if um you’re chased by heat seeking missile you wouldn’t feel calm and relaxed and be like Oh I’m not worried because a this machine can’t have goals in some deep philosophical sense you don’t care about that deep philosophy you just care that it’s acting as if it has a goal that’s where the thre comes from right and if you have if you say to yourself oh you know AI will be become kind and so on just because it’s smart so let’s just build some smart machine and have it figure out everything for us no you know if Hitler had been smarter things wouldn’t have been better things would have been worse so it’s for reasons like these that so many people on touring through the Sam Alman have been warning that this this could really be game over for Humanity could be lights out for all of us can cause human extinction or just some kind of orwellian permanent disempowerment of of humanity and last year who’s who of of AI thinkers came out and said this could even be something that could maybe cause human extinction and I have to give a compliment to companies for spending at least um some of their money on trying to solve the control problem but honestly they’re nowhere close right the the biggest success so far of these corporate efforts has been um training large language models to not say harmful things as opposed to not wanting harmful things that’s like successfully training a serial killer to not say anything that reveals their murderous desires problem solved right and it’s even worse than that because I think it’s getting increasingly obvious now that the F the first AGI if we get one soon is not going to be a pure large language model which is the context in which most of this safety work is done it’s going to be some kind of hybrid scaffolded system maybe an ensemble of different LMS interacting with each other and interacting with a bunch of tools and other AI techniques and if something like that is recursively self-improving we really have no idea still how to control them but you might say who cares if we can control it it’s desirable to just have this new um successor species or whatever you know we had only two or three hands support saying you wanted something that replaced us and I suspect some significant fraction of those two or three people were you were joking because you like to be contrarian but there but there are let’s face it a number of very serious and respectable thinkers who actually want this who want uncontrollable AI for example I really like Professor Richard Sutton you know F amazing pioneering work in machine learning U and he uh you can see him on YouTube talking about how he feels that that U this AGI that we lose control over is just a natural next step in in evolution and we should welcome it you see um Beth jizos who founded the eak movement that many of you have come across on Twitter tweeting out here how he thinks it’s just fine if all biological life gets replaced by machines and his his great fan Mark andri has this tweet here where he says um that the AI is going to be gloriously inherently uncontrollable so if you lose if you lose he actually thinks it’s good and uh you know you in your show of hands it was clear that Bas almost none of you agree with this perspective but you know these people have a right to their own opinions but so do the rest of us and uh I I think of this um school of thought as digital Eugenics basically craving to to to replace ourselves with some sort of digital Master race you know and although they’re entitled to their opinions you know that does doesn’t mean we have to share it I personally don’t share it I would really like my 2-year-old son Leo to have get to have a meaningful and long future even if some digital someone else is drooling over the idea of a digital Master race I am sucker for democracy I think everybody should have a say in this and we shouldn’t just let some Fringe minority decide all this stuff behind some locked doors or in a in a boardroom somewhere but you might say AGI is inevitable anyway so just shut off Max sit down and you know don’t waste their time fighting against the inevitable actually it’s not inevitable you might think it’s inevitable that you’re going to get every technology that can give money and power but the the data shows this is not true we have banned many many powerful Technologies that you could get very rich off of if I could make a lot of clones of Yu yosua for example in human cloning I could make a ton of money but we banned human cloning uh human Journal line editing U we also decided just not do because biologists felt this was a little bit too risky we might lose control over the future of our species buy weapons can give you great power we ban them so if there’s enough stigma around something then we will as a species often just decide hey we’re not going to go there and um if you um so if you meet run into someone here in Europe who’s like very red pilled on AGI and is like we must have AGI immediately you whatever give them a little bit of a hard time and and maybe remind them that if they have power or agency that they appreciate the they stand to lose it all if we rushed into this too fast and also so if someone still tells you it’s inevitable that we’re going to get AGI just imagine that that um they would go into the FDA and say Hey you know I have a biotech company and it’s inevitable that 3 years from now I’m going to release this new super drug in super markets and and I hope you guys at the FDA are going to like figure out how to make it safe in the meantime maybe you can fund some government some Safety Research you know they would get laughed out of the office like say dude where’s your clinical trial oh you don’t have one come back when you do next please you know so if we have some safety standards as a society which we actually have for every other industry then automatically things releasing unsafe things is not inevitable it becomes the other way around but China I’ll save save the best one for last because this seems to be the most effective argument South atast South of the Border here for ending all conversations about about uh not racing to AGI even yeah maybe we’ll lose control over it maybe it has all these risks but we have to do it anyway before China does this argument is of course completely misramolai AGI at all cost before China does I call or this this is a battle to race to to AGI I call it the hopium war because it’s fueled by hopium you know this delusional hope that we already know how to control this stuff why because let’s face it we are closer to to build figuring out how to build AGI than we are to figuring out how to control it so that means that the AGI race is not a traditional arms race it’s a suicide race just like in the raise your hand if you saw the movie War Games Once Upon a Time strange game the only winning move is not to play Strange game the only winning move is not to play if um you know if if if China builds super intelligence and then loses control over it then uh after that the Chinese Communist party is not going to be in control of China so they have no incentive to do it if they believe that they’re not going to be able to control it and vice versa here in the west right yet it’s very often misconstrued and described as a very traditional arms race where they sort of the US government had a report out recently saying we should race to build AGI and they just took as an axiome that of course we can control it I was in a debate that Rand corpor um last week or week before that against the authors of this report and I asked them why didn’t you even mention that there is such a thing as a control problem that some scientists have actually talked about why wasn’t there even like one sentence about it and they we didn’t have space for it in the report do you know how long the report was 760 pages so with these six buts I mentioned I I tried to make the point that U none of the objections to what yosua and I were were pushing here are particularly particularly solid so what do we do instead then let me spend my final um time here outlining my optimistic vision for how we can have a really great future with much much better and more awesome AI that we stay in control of so as I said in the beginning it’s a false choice to think we have to just have completely unregulated AI with no safety standards or nothing and go back to being Amish or something if we have if we treat AI like any other industry and have safety standards in place there’s going to be huge incentives for companies now to meet these safety standards and innovate and so same for us academics and we can have a really great future there’s a nice long document called a narrow path put out by control that uh suggests a a policy vision for this that I I think is really good I’ll give a little more of a colorful summary of of how I think about this so there are two things that have to happen on the one some on the policy side some on the nerd side on the policy side as yosha said governance is very important and uh the way I see this playing out is if if once it becomes more clear for more nerd research that the policy makers that the control problem exists and we have we’re nowhere near to solve it then both the US and China will just unilaterally enforce national Safety standards again AI is the only industry right now that has no safety standards at all in the US even if you want to open a sandwich shop there are safety standards you need to have the health inspector come in first and see that you don’t have too many rats in the kitchen you know so us will impose some safety standards and not to appease China or anything but just for to protect Americans and China will do it independently when the FDA was created in China and in America they didn’t have a treaty inspecting each other’s fdas they both did it selfishly so no coordination is needed there but then once that’s happened there’s a really interesting geopolitical Dynamic that plays out because now they’ll be like wait a minute how can we make okay so we we know our companies aren’t going to make AGI anytime soon because they don’t know how to control it yet but how do you make sure North Korea doesn’t do it so now the US and China have an incentive to jointly push the whole rest of the world to join them in having comparable safety standards and I actually think that conveniently the US and China together have enough geopolitical clout that they can pressure North Korea and every other country into joining them in this and and at that point we enter this unprecedented phase of global Prosperity powered by by safe non- agentic and and safe tool AI that I’ll talk more about a real golden age where the nurses are just going to be ever more fun to go to every year all the fun none of the angst and then on the technical side what needs to happen there once you have these safety standards right the incentives in companies and universities change a lot if you go to buy to a AI company today a lot of um Engineers feel that the people on the safety team are just these annoying whiners you know are slowing stuff down and worrying all the time if you go to a biotech company that’s not at all the attitude that people there have about their clinical trials people they love their clinical trials people because they see them as the ones who are going to get them to Market Faster by succeeding to make things safe so as soon as you have safety standards the Dynamics Chang companies leading AI companies spend much money on AI safety and the people in the tasty team feel really appreciated and there’ll be a massive amount of innovation what kind of innovation am I most excited about so Yoshua and I and a bunch of colleagues here were on this paper about guaranteed safe eye safe Ai and you heard a bunch of it those of you who were here earli this morning from from yosua where you actually have quantitative guarantees or other reasons to believe that the systems are going to do be beneficial uh guaranteed saf I mean guaranteed safety can mean that you have a quantitative guarantee like we have today if you want to build a nuclear reactor outside Vancouver they will ask you to guarantee to make some calculations and show that the risk of a meltdown is less than maybe one in a million per year it’s not zero but it’s it’s low enough it’s the same with drugs you don’t have to prove that nobody ever is going to have a side effect from the drug but you want to have something quantitative uh now the most extreme kind of quantitative is if you if the probability of a bad thing happening is zero that’s what Steve omah hund and I we’re talking about here in um in our paper about um guaranteed safe AI sorry provably safe Ai and the idea there is that um you know just as with with physical systems we put in place guard rails right to to to minimize harm but these are not impenetrable if you uh so the ultimate sort of guarantee is if if you have a mathematical proof that your code or your system isn’t going to do certain things because then no even if the AI is way smarter than us it still can’t do the provably impossible and uh I want to add a bunch of optimism here so far the whole field of proving stuff about computational systems known as formal verification it’s been around for a long time and frankly I think it’s been kind of disappointing it has very little uptake mainly because it’s so hard to do it takes it’s so expensive to do and very human labor intensive but this is all going to change because of of AI I think it’s quite obvious that that just as AI is swept in like a tsunami and disrupt did our ability to to generate art generate text and now generate software raise your hand if you ever use the copilot or something to accelerate your work right we’re sort of in the beginning of a huge huge SE change there AI will also obviously revolutionize our ability to make proofs for stuff we’re beginning to see it already and proving stuff about math and it’s clearly going to happen for proving stuff about code I’ll come back to that in in a bit of De more detail how we can further accelerate the adoption of this but I wanted to share another optimistic thought with you first I’ve worked a lot with my MIT group on mechanistic interpretability and trying to figure out how large language models actually do stuff and many others in the audience have too there’s a workshop next door on it today on interpretability and it’s hard it’s very hard to prove a lot of things about today’s most popular architectures but maybe you don’t have to for a lot of really cool tools you want we have to I think let go of this idea that just because we use AI systems neural networks to learn stuff we also have to have the deployed tools you have neural networks in them I think that’s a mistake like suppose for example that the yosua Benjo comes up with uh disc he thinks about hard and comes up with a really cool algorithm better algorithm for navigating a rocket for the control of a rocket that’s going to go to Mars you know and you it’s really important for you that this rocket doesn’t crash or something because you’re going to send your your family on it you know one approach which is kind of the current default approach people assume is okay you put yosua Benjo in there to steer the rocket and then you’re like oh let’s formally ver by yoshua’s brain let’s figure out how his brain works and try to get provable guaranteed does that seem really easy and feasible no right what would we do instead we’ be like hey yosua you know I hear you came up with this cool new algorithm you know can you just code it up for us and C++ or python or something he would do it and then we can study the code in fact we can form try to formally verify the code and get a rigorous mathematical proof that it works that’s clearly much more feasible right so to back up a little bit here what I’m saying is if you compare neural networks with more traditional forms of software it’s obvious that neural networks are much better for a lot of stuff but what exactly is it that they’re much better at is it that they’re much better at runtime to just execute the algorithms no actually a neural network is a massive a very efficient maximally massively parallel computational architecture right to in complete but hey you know so is massively parallel C++ and all sorts of other languages right new networks are not worse particularly but also not particularly better than traditional software if you’re just executing known algorithms so where is it that neural networks are so much better it’s for the learning that’s where they shine and the real superpower of neural networks doesn’t come from inability we don’t understand how they work it comes from the differenti ability that you can always compute any any set of parameters is Val valid program so you can compute gradients and use them to automatically learn the algorithms they can learn all these algorithms that we humans hadn’t even thought of good luck taking a gradient of a C++ program to see how you should improve it right so in other words what I’m saying is a a paradigm that I’m actually very excited about is in as we use powerful machine learning systems that we don’t trust to invent algorithms discover algorithms discover knowledge we need and then code it up in more traditional languages and then we use if either we ask the same AI system or a different AI system to also find a proof that that this this AI tool meets the specs that we humans have written down and you might say oh this is hopeless obviously I don’t understand how this powerful AI works I don’t I can’t read the tool it’s the code is too long and messy I don’t can’t read the proof it would take your lifetime but here’s the really great news just like it’s much harder to find a needle in a Hy stack than it is to prove to verify that it’s a needle after you found it it’s much harder to find an algorithm and find a proof that it meets your spec than to verify it once it’s been filed right in fact you can write a 300 line python code that will checked any proof for correctness so it’s perfectly feasible for you the human to understand the spec you wrote and understand the proof Checker and now you can suddenly start trusting something a super powerful tool which was written by a completely untrusted system it’s a bit like if some f if some mathematician you never heard of who was convicted twice for fraud comes up with a proof of the reman hypothesis you know you don’t have to trust the her or him as long as you can check the proof and that can be automated something that yosua has emphasized a lot which I think is very important is that um we want Technical Solutions that scale in the right way so that they the Technical Solutions become more effective rather than less effective as AI gets more powerful and what I’m push pitching here it has exactly those properties because the better the more more powerful AI becomes first of all the better it’ll become at discovering new algorithms and also writing them up as in code and the better it’s going to be also at coming up with proofs so I so I think it’s super interesting to work on this stuff now scale it up and then watch how as AI gets better ASM totically the gap between how how much how awesome and how powerful these AI created provably safe tools are and what unsafe things are will I think will shrink as things scale if you can’t figure out how to um how to um and even if the I cannot in a traditional way itself write down its algorithm it’s figured out you can also have another approach where you have an AI neuroscientist so to speak that looks inside has access to all the weights and tries to figure out the algorithm and cod it up a little toy example to just show you that it’s not possible is this paper we recently did where we trained the neural network to just learn 63 different numerical algorithms from data and uh then we wrote a fully Automated machine Learning System that looked inside at the weights figured out what algorithm was learned and distilled it into python code so we could throw away the neural network as a simple example we did an algorithm you probably learned in first grade or kindergarten or something for adding up numbers where you Loop over the digits and and do a Carry operation and um so we would train an neural network to get perfect accuracy and then we would what what we what what the automated system found was that in a certain latent space after a while it looked more like a finite State machine and in this particular case it could be interpreted as two bits and then by looking at the finite State machine tables Etc one could figure out what it was actually doing and It produced this python code completely automatically and it’s rediscovered the so-called Ripple ad algorithm here which we is actually the one we learn in first grade where it turns out that the the B variable there is the carry bit etc etc and we also saw many examples where we noticed that it’s neural even though this latent space is continuous it’s representing integers as these laes and we haven’t came up with an algorithm for taking arbitrary incomplete uh integer lattice which was messed up by naine transformation and figuring out what the integers were there’s a lot of exciting progress in this field both the previous speaker and the next speaker very excited about it and we were working together on it and in summary um formal verification I think can really be scaled up in a in a bague way and I want to end in my final four minutes here here just talking about um some imminent opportunities for accelerating this this process a formal verification so I think there’s a great opportunity to make this progress faster with open source community building the vision here is that so right now the adoption is very low so there isn’t very much formally verified code so there aren’t any good benchmarks and training sets that train machine learning on if we can get larger benchmarks then of course M folks like you will train you will train on them to make better automated provs and that in turn will produce tools verification co-pilots right that makes it much more fun and quick and cheap for people to do formal verification of code and then we get larger code bases we larger benchmarks and we can go around this this virtuous circle right now the the Comm things are really very fragmented in this field a lot of people who have worked on formal verification for years don’t know much about machine learning and don’t have don’t know a lot of machine learning people there’s a horrible lack of large benchmarks and the code the people WR is very fragmented too some some of the biggest hits and successes in formal verification are each done in a different language Amazon web services just completed this massive project where they formerly verified their whole login and credentialing system so they will never have to do another security patch ever they did it in a language called Daphne the greatest math Library that’s formerly verified is in in lean the first ever formerly verified C compiler compcert is done in [ __ ] when they did this they found over 100 bugs in the previous C compiler so correct C code gave incorrect machine code like the first formerly verified micr kernel was written in Isabel Hol and the simplest proof Checker is in a fifth language called metamath and and there is no automatic translation between these so to realize how annoying this is it’s like suppose a FR French mathematician proves some cool theorem and an American mathematician proves another cool theorem but they can’t ever be combined by anybody because nobody knows how to translate between French and English so let’s look a little bit more the sad state of benchmarks here for mathematical theorem proving we have many benchmarks now with a order 10 to the five examples of of U theorems that you can have contest people try to prove and as a result the ability to automatically prove these things has gone up to from success rates like 2% of those benchmarks like over 70% for formal verification the biggest benchmarks a year the beginning of this year was 66 programs in one of and 153 on another so much room for improvement here we did one recently where we at least took it up to 750 examples in this DNE bench paper spearheaded by Wendy sun and and Khloe lowbridge we basically took formerly verified code from GitHub where the there’s an algorithm and it’s supposed to meet the spec there and blue at the top and then we threw out the so-called hints that humans have to put in by hand to make the proof work and asked if and then we asked the standard off the shelf llm is to see if they could put back enough hints that the proof would work so it could all be done without human input and even with this minimal effort we we get like two-thirds of them succeeding um I hope you can beat this this performance with with better algorithms and help make bigger benchmarks and then on the community side we just launched last week a website called very.org it’s an open source project for formally verified code where you can go upload your stuff and we have proof Checkers in the background so if you come as a user everything you find here is guaranteed bug free you don’t have to trust those who wrote it because been automatically checked and we hope this site will help uh basically get this virtuous circle going faster it’s like a guaranteed bug-free version of hugging face to summarize I’m an optimist I think we can have a fantastically inspiring future if we just impose some national Safety standards which countries don’t have to negotiate about to trust each other first and um incentivizing all sorts of wonderful AI research so that we can basically have our cake and eat it we can continue going from neps to neps with new cures for diseases and all the other wonderful stuff we want without having to constantly worry that someone is going to ruin it all by creating something that we lose control over in other words just to summarize we just have to remember this lesson from ancient Greece and not get hubis if we have we have a very long future for Humanity if we don’t do something dumb so let’s not rush it and Jinx things artificial intelligence has given Humanity incredible intellectual Wings which we can use to accomplish things beyond our wildest dreams right if we heed this warning from from the myth of Icarus and stop don’t get hubis stop obsessing about flying to the Sun and building AGI as fast as possible thank you [Applause] we have some great so we still have uh quite some time for uh several questions um please raise your hands if you are interested in like uh and this one so uh you have microphone we can pass them out okay I should ask maybe ask these ones work if you can turn them on oh my God we have we have microphones here should we use them yes start from mine works I just wanted to give him one yeah so we have oh fantastic oh good oh oh I didn’t see that I didn’t either so where is it over there around St see where the gentleman is waving his arms that’s where you should go to ask questions yeah you can take it away yeah since I have the mic um thanks a lot this was a very interesting talk uh I’m pushker from Deep mind and I really uh love this idea of having guarantees and personally I’m a big fan of uh air crash investigations and um it’s very interesting like uh what you can learn from the airline industry on how every part is verified to not fail in a certain number of takeoff Cycles or Landing cycles and every part comes with like an operational guarantee that we if operated Within These uh paradigms the part will not malfunction and um I just wanted so I have a comment and um just a question the comment is that uh and maybe you have already thought about it the comment is that there’s definitely guarantees and we should be able to uh provide those guarantees for different operation models uh or different modes of operation of a model uh the other aspect of this is maybe having a proper uh AI accountability frame work for when we cannot have these guarantees or when the guarantees are not very clear um so that’s my comment if you want to respond to that first or yeah maybe just say everything together so we can fit in one or two other questioners also yeah and the second Point uh is I don’t know if you’ve already thought about it but uh an interesting aspect of this is what do these guarantees and uh accountability framework look for foundational model like system level uh implementation versus uh Foundation model level implementation uh and so on great I think I’ll answer the two together I think uh aircraft is a aircraft industry is very inspiring you can see having safety standards has taken the fraction of planes crash from being like one in four or so right after the right Brothers to making it ridiculously safe the probability of you dying during 1 hour in an airplane is not only less now than 1 hour in a car but even then what less than one hour in your home it it turns out it’s amazing um and and so these these incentives are incredibly important and for for the second question I think it’s the wrong framing to say oh ask some random Professor on a stage or some governance person to figure out how to make things safe soon as you have safety standards you provide wonderful incentives for the companies to innovate and answer all those questions and I I have a lot of friends at Deep mind and I don’t want you to take this personally but I do think it’s disappointing that deep mind has used their lobbyists to to try to push against regulation even though we really need some safety standards very quickly so as soon as we can just rally around this idea that we should have some safety standards which can which can be very low a very low bar initially then I’ll feel way more optimistic uh next and let’s make sure this is a just a question so we can have time for quick question so you say you’re optimistic but then you also say that that 1 to 10 years we’re going to have AGI might we might have AGI companies are racing they’re saying right vocally that they’re Racing for it and governments move slow right governments are not very fast at putting in regulation yeah how are you optimistic given all those facts you’re hitting exactly the Crux here of the question we will get a quite it’s quite likely we will get AGI soon I think if there are no safety standards but that’s exactly what the problem is it’s like if we had no safety standard for biotech I could get it’s very likely we also will have some very unsafe medicines within the next 10 years but we don’t worry about that because we have an FDA Etc and so as soon as we can have some safety standards in place then uh it’s pretty clear we’re not going to get going to get AGI next year because no one has figured out yet how to control it if someone can figure out how to make it really safe and convince everybody all the experts that meet the safety standards in 20 years yeah maybe we’ll have it then but in the meantime we’re not going to nobody is going to be allowed to release un something they can’t control but in the meantime as both yosua and I have argued here we’re going to get wonderful other AI Innovations fantastic non- agentic AI fantastic tool AI maybe a cure for cancer remarkable autonomous vehicles and and amazing free education customiz all this other great stuff right will happen happen so it all starts with just putting in place safety standards so that the the Nerds of us here can just get back to doing research on good things and not have to worry about there being some kind of Rush the whole stress we feel oh my God are we going to figure out how make it safe and before someone releases this it’s just complete nonsense we there’s no other industry where we lack safety standards and I have to stress like this hello uh on your right this is the last question yeah so um oh oh you are the boss they you’re saying that there’s someone here oh can you okay so this is about uh what you were talking about earlier where you were saying that you want a Jo joint initiative from like China in the US to ban unsafe AI so I I feel like you didn’t really touch on how we would go about actually enforcing such a policy and what would be like the timeline of that because I feel like most people okay go ahead go ahead great question so framing is crucial here I think it’s very unrealistic that the US and China this year are going to start hugging each other and trusting each other and so on U so my very optimistic point is you can have some really great things happen without even before any of that right uh if as soon as the as soon as people in the US natak community and the US government start to realize that we’re closer to building AGI than to figuring out how to control it they will put safety standards in place in the US they would do it even if China didn’t exist same in China they will put Cy standards in place there just like they made a Chinese FDA right doesn’t require International coordination and it’s enforced individually by the separate countries if we have some kind of FDA for AI in Canada and another one in the US you know they’re locally enforced that can happen as soon as politicians get start taking it seriously and uh and then after that if you look officials from the Chinese FDA and the US FDA they like to talk to each other a lot about sharing best practices and harmonizing a little bit so it’s easier for a company that gets licensed in one country they sell and the other that kind of takes care of itself but I think it has to start very quickly from the Grassroots level and then it can start with us if we just whenever we talk to policy makers just remind them that that you know there’s more safety stands on sandwiches right now than on AGI and if if one of the companies had AGI tonight they would be legally allowed to just release it tomorrow and see what happens that’s nuts thank you and I can take more questions in the coffee break from anyone who didn’t get to ask it now thank you