FOR EDUCATIONAL AND KNOWLEDGE SHARING PURPOSES ONLY. NOT-FOR-PROFIT. SEE COPYRIGHT DISCLAIMER.

Dominic Cummings (@Dominic2306) on the alarming naivete of those calling for the US to race to superintelligence: pic.twitter.com/XPnphUP3OV

— ControlAI (@ai_ctrl) December 17, 2024

Dominic Cummings: is AI already in control? The Spectator

TRANSCRIPT. so thank you all for coming uh I’m soft spoken but but but the good news is Dom will be doing most of the talking and he has a a voice that will project so um just a few things at the start so as miles mentioned um I thought it was a time where it would be instructive to have a sequence of conversations about the intersection of AI with various domains of of Human Experience um when I myself think historically U you know I tend to think of naturally as a scientist of advances in science advances in technology the ordering is often unclear uh but you know but then so that’s the sort of deepest driver but then you get to economics you know maybe that takes you know 50 years then there’s that tends to shape culture and then but on the other you know but then if you get down to an even shorter time scale there is the classic subject of politics about power and its exercise and and how that plays out so I thought it would be interesting to have a discussion on where things might go on a political time scale which you can think of as 5 years 10 years something on that range the other reason this is interesting is that even though no election to date including the recently concluded American election has had the advance of AI as a major theme in fact if all you do is read what some of the heads of major AI labs are saying in their public writings and interviews you have to wonder why this is the case so to take just one example Dario amade who by the way was a graduate student in physics at at Princeton but I was director of Graduate Studies um may end up being the most significant uh PhD in physics from Princeton uh you know for a while maybe forever um he wrote an essay saying that he expects that human level performance across the board is likely to be overtaken by stuff going on in their Labs as early as 2 years from now right maybe and that it will be followed by the proliferation of such superhuman or at least postum int you know extra human intelligence systems to be followed by perhaps a 100 years of science coming by in about 5 to 10 years right so that’s if you think of this happening in two years you really have to wonder where we are for the purposes of today’s discussion Dom and I agreed that we would take a much more modest Baseline we would somehow imagine that something like this will happen over 10 years I have a graph here that you can’t see which has a straight line of sort of you know competence level versus time and sort of a genius level thing at at the at the top for humans and this idea that in 10 years from now you’ll achieve a Crossing 10 years is still remarkably short on human scales right the generation is 25 political Cycles are 5 years you know so this is the sort of mismatch between you know chemistry based life forms which is what we are we are and you know sort of electronics based you know the electromagnetic interaction versus chemistry right the these These are different time scales now with that comes this notion that unique human skill hours so I don’t want to say people will become obsolete but particular skills will become obsolete some fraction of the time I teach physics honestly even today if you ask gp4 fairly sophisticated questions in physics you know you can get answers which are not bad most of the time uh the reasoning models again publicly available will do stuff that entry level graduate student might do in terms of take the following differential equation you know tell me if this is true and you know you can it just tells you I spent 11 minutes seconds thinking about this and so on so all of this stuff is you know is is sort of visible so you could ask well what’s the you know let’s talk about remaining unique human skill hours and you can imagine another curve maybe let’s say it’s here and there now and over the same 10year period you know that’s going to decline so view that as a backdrop to the conversation that we’re about to have this is apparently about to start happening and there’s just no way that this process proceeds without it intersecting with the political real now I had asked two people uh Josh Simons unfortunately had to cancel because politics involves especially he’s an MP there are some farmers who are unhappy with the starmer government in London he had to stay and um either deal with them or or so on so he was uh very unhappy he couldn’t make it he by the way uh did a PhD from the Harvard’s Kennedy School of government he wrote a book on AI so he was an obvious candidate we will ask him back Dominic read I think around 2000 Bill Joy’s famous essay uh called I think the future doesn’t need us or yeah that why the future doesn’t need us I think something like that and so since then even though he’s not a scientist by training he’s been absolutely convinced that technology will shape the future uh and if you read his copious writings which I can’t say that I’ve read all of them because they are copious but I I do recommend he’s been thinking about technology the generation of Technology what lessons it has but also just you know the future for a long time so really uh he’s I know he’s known for many things in his actual practical political skills which could be the subject of a completely orthogonal conversation but today he’s here as someone who’s thought deeply about the Arc of history and how it intersects with politics so that’s the background the way we will proceed is we have some things we’ve sort of decided we will talk about we’ll talk about and we will leave time at the end for all of you to join in uh and of course there’s a reception at which point you know he’s available for uh General discussion so so with that um maybe I can so start with one high level question which is that we could ask the sort of question if all of this is going to happen what might be a desirable trajectory for societies and humans and what is a likely trajectory so pick whichever one you want and and you know start with you know what do you think these what challenges do you see coming you know uh coming up as as as these what are going to be the signal events you know how will people really notice that this is going on um well I think like for sure it’s it’s inevitably going to be just um unbelievably chaotic I think there’s no way around that if you look at um how these how technologies have interacted before with human events if you look at things like the railway and the telegra telegraph coming in in the 19th century um or later on with mass media these big technological shifts always lead to huge changes in power they’re unbelievably hard to to predict how they’re going to work it’s very hard to know what things are going to be centralizing and what things are going to be decentralizing um you know just to give one obvious example at the moment it seems to me that I was reading recently some accounts of French peasants who were drafted into the war um with Napoleon and they were kind of thrown across the continent to Moscow came back and asked all of these questions and you can read them now their answers they basically had no idea what on Earth the whole war was about the whole thing was completely baffling to them there was no real consensus reality in kind of 1800 Europe because of the media and information structure and Technology structure if you go forward to to a period like 1940 you have a time when um people like Stalin or the head of the BBC or the head of one of the big American um radio and then in the 50s TV networks could kind of determine consensus reality for hundreds of millions of people Stalin could make certain kinds of decisions he could take people out photos he could dictate that the Soviet encyclopedia will say blah blah blah so for a brief period of time you have this period where a very small number of people and a very small number of very centralized institutions exert this great power and there’s all kinds of consensus reality amongst Elites and amongst the public what’s happening now it seems to me is that the world is already Poss arguably now but if not heading very quickly towards situation where it’s going to look more like 1800 than 1950 and you can even see this just over the last sort of 10 years ago if you go back and look at um people like Obama and Elon Musk and editorials in the times and all these sorts of things you go back 10 15 years there’s a clear sense of kind of like consensus reality among Elites but if you look at it now that’s all just completely fragmented in the in the last few years you have people like Elon and a whole bunch of that that kind of network in Silicon Valley who regard the mainstream media as essentially like either mad or actively malign but in any way obviously not something to be trusted not something which defines reality for them and you have those same old institutions which were extremely friendly towards people like that even 5 10 years ago saying that these people are essentially fascists deranged crazy pushed even to the sort of extraordinary extent to which it’s become completely accepted con conventional wisdom amongst political journalists in Britain to give their views on Elon as as a tech company manager so over the last three years they all tweet away happily saying that Elon doesn’t understand tech companies he’s completely useless meanwhile Elon has catching skyscrapers falling from space with chopsticks so you have this very very odd situation happening um I think the same thing has just happened with the election right again you’ve had this they’ve had the same experience that we now see repeatedly political Elites just being hering towards opinions those opinions um smashing into reality them being completely confused by what’s happening and then they double down so we saw that with brexit in 16 we saw it with Trump in 16 we saw it with when me and my team were in number 10 in 2019 we’ seen it again with Trump now seen it on Ukraine seen it on on on covid you just have this repeated process whereby the political and mainstream media Elites who think we’re the the sensible ones the public are idiots fooled by disinformation and conned by the sort of you know these tech companies what not and and a growing counter Elite emerging saying actually we’ve it’s the old political systems that are the problem these things are like irredeemably corrupt irredeemably useless we actually need to build new things so if you look at these Trends I think um almost inevitably I think it’s going to is going to continue to be chaotic the political world is absolutely determined in not to change in all sorts of ways um uh certainly in Britain Washington’s more complicated um but in in Britain um we’ve had the biggest pandemic in a century and we’ve had the biggest war land war in Europe since Hitler and both events have have led and brexit of course before that and all three of those things led to an extreme determination of the governing systems inside Westminster to say we will not compromise electorate we will not change Tac and we certainly will not start changing how institutions work to absorb how these Technologies are changing in fact the ex the exact opposite so one very simple story to to to illustrate it in 2020 um I brought in various people from the worlds of physics data science Ai and whatnot and built a team in number 10 which brought in Elite Talent to the PM’s office we started doing this in January actually before Co got going but obviously Co gave it a huge turbo boost that had a very dramatic effect it led to all sorts of things working immeasurably better you actually had uh extremely able technical Talent sitting 30 MERS away from the PM and available to give the PM advice on day one of the new government going in and K St arriving in number 10 one of the first things in pure classic srey fashion happened was he was obviously given lots of documents to sign one of the documents was to sign off the destruction of this team and this unit so the data science and AI team that was built to support the Prime Minister the concept which has been copied all over the world is as we speak being dismantled by the cabinet office the people have all been told that they’re not welcome they’re all being pushed off etc etc etc so that gives the sort of um uh I think a picture of the kind of pathological nature of these old political institutions now even as this revolution is happening they are they see Power as a sort of zero some thing and they don’t want to open up any of these political institutions to to these tools and to the people and to the talent because fundamentally it’s threatening the core of the cabinet office and treasury hate of this is information is power and if you have a situation in which the Prime Minister has access to the best information and tools that we can’t even use and don’t even know how they work well that completely upends the power relationships right and the the way the British state has worked for a long time is that the Prime Minister office is not actually in control of the government the cabinet office and the treasury are in control of the government hence their destruction of this team because they want to make sure that the old the old ways continue so let me maybe try and theorize a little bit about what you said in the context of this so is it fair to say that some of the political chaos which is not solely in Britain by any means America is very polarized um Israel actually you know until the war and so on is that not itself sort of a result of this because the information revolution really dates to the ’90s right we’re now getting to AI but we’re already 30 years into an enormous expansion of information flows is it the case that the technology is preparing the way for itself because it’s making it impossible for normal political processes to work obviously what you’re suggesting is there are better ways of doing it but so yeah so I mean maybe if you could speak a bit to yeah I think so clearly there’s a lot there’s a lot of these things are happening which are not um not at all limited to a particular country a particular regime and Co is a very good example of that in all sorts of ways the the um Western bureaucracies failed in very similar ways basically everywhere um but also uh subsequently they also behave very similarly almost everywhere so so two very simple obvious examples two of the things that were most successful in Britain were a we created the vaccine task force to try to figure out how to do vaccines much faster than previously was done and secondly we created uh A system that brought together uh Wastewater monitoring and AI people mhm both of them were closed down in 2021 rather than reinforced what obviously should have happened is co task force was on what obviously should have happened is um Wastewater sewage monitoring was was expanded instead they were they were both closed down because they were both seen as so um threatening and um uh uh to White all so much so that after an outcry the C office tried to rebuild the the Wastewater AI system but they said to the people that built the first one you’re not allowed to bid for the process to rebuild it with the same people that built the first one because that means that your bid would be really fast and really cheap so that would be unfair on Del and all of the our normal suppliers so you’re you’re the only people banned so we the people who actually built the system which was then copied from South America to Asia they were explicitly banned from rebuilding the thing that was vandalized in the first place sounds could of completely crazy but very similar things happened in other parts of the of the world so for so operation WP speed was similarly closed down in America and that and that’s my point that these pathologies I think are that there’s a set of as you say technological things which are disrupting all these regimes and the economies and information in the old media old media is tanking at the same time subscribers and everything are collapsing um but also it’s creating these path but also the pathological institutions are failing in very similar ways so it’s not just a British thing that’s the the core patterns you can see everywhere okay so maybe we can summarize this part by saying that you are highly pessimistic that existing States will be able to cope with this I think so if you just you know if you just look at how they cope with say gain of function okay now think of how they dealt with regulat regulating gain of function before the pandemic was already completely insane and right and disastrous you’d have thought that after covid that would be a bit of a wakeup call to get a grip of Gator function no no no no no no no no no of course not so we’ve continued to have completely mad attitudes towards this kind of research we’ve we’ve continued to have the CDC and FDA in America vandalize a lot of actual inspection of these things we’ve seen massive coverups of um in bureaucracies Across the Western World in terms of how they handled all of this and what they funded so again you see this this kind of now gain of function relative to AI is like trivially simple right it’s like it’s it’s a matter for like expert people to sit down and look at and try and weigh up what the pros and cons of it all are and then how to regulate it and then how that actually should be done um it’s uh you know it’s relatively just extraordinarily simple relative to the extraordinary issues with AI where there just a whole bunch of complete unknowns that we can’t even sensibly way up and the greatest experts on the planet who are actually building these things completely disag agree about so if the old pathological political institutions can’t cope with relatively very simple things like gain a function and just have even even have honest debates about their incompetence about it is beyond them then it seems practically impossible to imagine them coping with this all right so from that though um let me segue to something else which is to say both Dario’s essay and there’s another one by the way by Leopold Ashen Brer who uh it’s called situational awareness which you might also look at so Li is a very you know the people you want to read at this point are all shockingly young as is often the case with revolutions so he was uh I think a superstar student at Colombia somehow managed to get uh an A+ on every course and even though there’s great inflation at American universities it’s still not easy to get an A+ on every course and he was I think leading the alignment team at open Ai and then he left and uh and he’s outside so both of them I have suggested this notion that as this AI explosion takes place and generates an enormous amount of latent power that the thing to do is for the United States leading the Western World to somehow raise ahead of China in the context of the current International situation uh obtain I think you know maybe a two-year lead which given super intelligent systems you know could mean the same as having a 20year lead or a 50-year lead and then negotiate from a position of strength you know with China so I guess my question for you is uh is this at all plausible that never mind you know to to zero the approximation of the question well you can address both is this is this a desirable course and is it at all possible that the states that you function that you were just describing the inability of governments to do basic task that they’re capable of bringing off anything like this so I think on the the the second question first um I think anybody who goes and reads some of the classic books um such as now it can be told by General Groves who who was oppenheimer’s boss um and ran the ran the Manhattan Project um if you go back and read some of these classic books about how the American state functioned decades ago how it ran Apollo the icbn project Opa Park Xerox and whatnot um that that America doesn’t exist and part of the reason that America doesn’t exist is that the PE the sort of people the general Groves type people are now are people like Elon they people like Patrick Collison they’re building SpaceX they’re building um stripe etc etc and they these people have um it’s part of a broader question but I think one of the key things that’s happened um and one of the reasons why we’re in the state we’re in is this huge Talent collapse that’s happened in Western politics um uh which is really really very dramatic I think historically because in the past there was always a a big section of the intellectual Elite and the kind of capable Elite that could build things was involved with political power I just been reading about the 1790s and 1800s and White Hall and it’s incredibly striking how the White Hall of 1795 was more like SpaceX 2024 than whiteall 24 is like SpaceX 2024 if you go back and read some of those stories about 1790s it has a very it has completely cican Valley feel people like pit as as prime minister and ministers called people in they promoted young Talent everything moved incredibly fast their motto was the mark andreon motto build build build they built extraordinary State capacity they had um the political community and the whiteall community uh completely working with priv private sector they did they took procurement incredibly seriously there constant debates people got jailed for stealing money and for screwing things up completely inconceivable now where procurement is uh complete and not a disaster and everyone gets put in the House of Lords and given massive rewards for failure in 1719 1800 we Lo we looked a lot of these um disastrous characters up so that’s one huge one huge shift the America of 1950 is not in charge in in in Washington but also I think that [Music] the some of I think leopo there’s a lot of interesting things in Leopold’s essay Leopold’s a thousand times smarter than me but I think a lot of these people have to go back and read some of the books that my old tutor read and I think that if you read fuses and the outbreak of the poonan war and you read his account of how that started and the escalations involved the idea that if America says well we’re just going to build this super intelligence which is going to render all existing military technology completely irrelevant but don’t worry because we’re the good guys and then from that position of strength we’re going to um explain to the you 1.4 billion Chinese how the world’s going to be structured but you shouldn’t fear this because we’re obviously beneficent and the world will just be happier if we do it like this and that the chin the Chinese are going to and go oh yeah okay that’s that sounds that sounds reasonable um uh okay we’ll get back to us in a few years time and we’ll negotiate or rather will accept your dictation to us about how the world’s now going to be structured that obviously is never going to happen in in a million years it’s not going to happen and China will take Extreme Measures and extreme risks in order to stop that happening because they won’t see this as some kind of joint benevolent um creation by the Western liberal democracies etc etc of a new world order which is to everyone’s Advantage they’ll see it as the latest example of Western world’s grotesque hypocrisy um and conniving nature which must be scuppered at at all costs um and they will try and Scupper it at all costs and therefore I think that um know you don’t have to go into a lot of the further out scenarios about what the capabilities of these systems might be we already know from looking at archives how things like the Cuban Missile Crisis worked out in a lot of detail we know for example we can read transcripts of JFK talking to the Chiefs of Staff in the Cuban Missile Crisis we can see the kind of assumptions they made and then we can now look at archival things and we can see all kinds of ways in which the conversation was completely wrong we never they all of the military advice in Cuba was for JFK to invade they had no idea about what was happening with the submarine incident that was Kept Secret for 30 years they had no idea that there were already tactical missiles nuclear missiles in in Cuba with orders to fire they had no idea that um Castro was writing to the Kremlin begging to have the war because even though Cuba would be completely destroyed it would be worth it for the Triumph of the Socialist Revolution everyone was completely stunned when all of these things came out 30 years later so we know like we know from history that we constantly don’t learn from history how Wars start and regime’s constantly confused and then are baffled when the archives are opened up and clutch their head and say my God it happened to us in 1914 it happened to us again with Hitler in 1939 with both sides completely miscalculating various things happened in Cuba um and if you imagine those same Dynamics happening in inevitably again but in an environment where the Technologies are nuclear destruction so like you know a million times worse potentially than than July 1914 and the time scales are 20 minutes okay that’s the current situation so you now have nuclear staff and 20 minutes time time decisions rather than like say two months in in July 1914 but then you imagine that already nightmarish problem being being overlaid with are these people even in control of these things is it them that are resuing this press release or is it the AI system that’s issued this press release do they even control their own cyber operations or is their cyber operation now like somewhat gone Rogue etc etc uh if you look at the debate over Putin and Ukraine without getting into the all the ins and outs of it one unavoidable thing is yet again complete miscalculation and and if you talk to the people in the Deep state in DC and in London it Remains the case that Putin is essentially a black box no one has a clue what’s going on in his mind or about about what’s happening in the immediate circle of power around him what their calculations are and how they would respond to certain kinds of threats certain kinds of things that um you know the chaos that inevitably happens with war planes get shot down is it deliberate is a mistake was it a signal was it complete blunder Etc so you imagine all these normal things that we know happened historically and then you over lay on that questions of the increased speed from these systems being automated and then the lack of clarity about who’s actually in control of what are the people running the system or is the system running the people it doesn’t strike me that the way to go is to um start another escalating crisis with China in fact I think we should go in the exact opposite way um I think we should everyone should read lequin news memoirs about this subject and should reflect that there was a bipar policy in Washington for decades which was there should there will be one China it will be United Taiwan will rejoin but it should be peaceful not through Bloodshed Biden blundered around changing that over the last few years but because he’s seile and gagar and says things that he doesn’t necessarily mean it was a very confused thing where he would say things and then it would be walked back by the White House and then he would repeat it and be walked back but finally after about four or five of these episodes China’s got the picture of hard you know he is sign cile so we can’t be exactly sure what he’s saying but clearly the attitudes are changing and if you look at the debate in Washington it’s obvious that the debate has changed there right there’s much more of a feeling of we can’t let Taiwan be United the the old consensus is gone I think it’s just another complete disaster and if we go down the path of America saying we’re not going to let Taiwan be unified that is heard in China as we intend to destroy China um and if you add the timelines of what Z saying on Taiwan and the Army the critical importance of semiconductors and TSC and then the AI timelines which obviously got all the recipes for um the classic catastrophe right well one one good thing just in the last few days on the positive side was that when she and Biden met they actually issued a statement saying that they had agreed that nuclear weapons would not be turned over to automated systems which at least is a recognition that they can see some of the challenges you know that you’re describing in terms of time frames we’ll have to see of course how how things go I think so one sort of connected thing to that though which I think is going to drive this this is um there’s a there’s a um you can see in Ukraine this uh kind of drone anti-drone um uh extremely competitive and extremely fast moving armor race happening in real time with a mix of classified Technologies and open commercial commercial Technologies and one of the outcomes of this is as people predicted 10 years ago would happen there is an incredible drive to autonomy and taking humans out of the loop because one of the things if you talk to Special Forces people who’ve been over there watching this one of the things that that obviously happens is on the one hand you have these things they you have people hundreds of miles away piloting them flying into this bunker and then blowing everyone Sky High but then you have um uh various kinds of defenses kicking in to take the drones well if you can just drop it in a vague area and give it an instruction and say find an entity that looks like the following and then make your own decision and kill within the following parameters right that’s all that is now already happening um so there’s a great the speed is critical in war and there and therefore speed is driving growing autonomy and therefore I think you’re going to see across this whole drone AI yes system this process remorselessly leading to Greater autonomy and less and less humans in the loop because the more humans in the loop the slower it is physics versus chemistry again yeah um no I mean if I look if you look at World War II it’s actually extraordinary What technological advances took place between 39 and 45 you know when you’re in danger of getting killed it really concentrates the Mind rather well and unfortunately absolutely so coming back a bit from The Intern Arena to maybe thinking more domestically imagining you know within a given Society um so we’re sitting um at new College a college and you know our selling point is excellent tutors educate uh you know the young people of the future um I think there’s a question to my mind which is we’re sort of going on as before we teach what we used to teach uh but in fact if someone asked me you know at age 15 what should I learn what will you know what kind of a career might I have in the future uh I wouldn’t really know what to say uh and I mean I mean that genuinely it’s not just me some uh physics used to have Joe may remember a series of online discussions in Ai and the former uh Chief technology officer of Microsoft was one of the speakers I I went back and looked at this and he talked about with growing enthusiasm of AI and AI training AIS and how there was going to be this super intelligence explosion and then finally one of the staff at the end said uh you know that’s all wonderful it was a great talk you know what should we tell our students you know what should they what should they do to prepare for this world so it’s interesting you know he you can see that he has nothing to say right which is to say because he really doesn’t think that there’s much advice to give but then he felt if he just said well their toast uh you know that wouldn’t go down very well so in the usual hou you know you say well you know my grandchildren they play with things that I never played with you know the general argument but of course it is true that humans adapt and have adapted to extraordinary changes in our circumstances but it’s a little hard to get around the idea that our retraining time isn’t years it’s often longer if you’re young maybe it is you know a few years uh and that ultimately it’s often from major technological revolutions that the Next Generation it’s your children who grow up in the new world but of course if the time frame for changes is less than 25 years right then you know as a physicist you know I always like to compare things so the time frame for retraining and the time frame for technological changes have crossed and these are two different worlds in some ways so I’m just wondering are we going to have a so let’s say anthropic and open AI unil in two years this amazing model which is a better physicist than most you know physicists alive a better mathematician maybe than most mathematicians alive and so on what happens then you know do people just say let’s go on um will it be like specifically specifically with education you mean yeah which is what will will will will the young the question of what is what are you to prepare yourself for does that does that become an issue at some stage does that generate or do people just go on um it’s possible that it almost vanishes as an issue mhm because I mean I’ve spoken to quite a lot of the kind of like if you think of like the top 30 people or so in the world who who deal with this technically mhm and they have very different views about it there’s clearly no kind of um there’s no sort of agreed answer MH the main thing one gets from talking to them is that these things are just highly unpredictable right there’s no real we don’t have any Theory as to why this machine Learning Works the way it does um scaling has just turned out to be an empirical fact for some number of years we don’t know when it’ll suddenly conqu out if it will conqu out I mean obviously has to conqu out eventually because laws of physics but um it doesn’t have to conqu out anytime soon um so in that environment no no one can really give a good picture of like what skills are going to be useful in 10 20 30 years time it seems a general sort of theme that people say is like it’s more likely I’m afraid that you will be redundant before your hairdresser um but other than sort of very vague comments like that people don’t really seem to have much much sense so possibly what will happen is that that it defaults to just people young people being told don’t think don’t don’t do education on the basis of careers and what you think is going to be useful and make you more money because it’s essentially completely unknowable what that turns out to be you could easily do something which we snook yourself um just do what’s interesting to you and what you love what you love spending your time on and if the revolution does work out is in the way that some of the companies think then you’re going to have a lot of leisure time anyway so um why not spend your time focused on what actually moves you and you feel every day like it’s just wonderful to be involved with so when 5% of the population at the right age went to college or had personalized tutors like John Stewart Mill and Burton Russell I I see that there’s that group but modern higher education isn’t that you know it’s 40% you know it’s it’s sold as Skilling yeah right most universities advertise you know uh courses and so on yeah so certainly getting back from a world of 40% college education to the 5% who maybe want to learn stuff for the I mean it’s again um that that so the main thing I guess is I can imagine end states which Nick Bost from our our Oxford uh uh colleague you know has a recent book in which he tries to visualize what what if things go right right so you can imagine far away there’s another equilibrium at which you know we’re all enjoying life and and doing stuff but the getting from here to there is a highly non-equilibrium process so to take an example uh you know uh yes certainly you know I’ll be out of work uh so there’s one problem with that which is to say I also have the skill set to make trouble right which is to say so there’s that thing about aliot over production yeah this will be aliot over production on steroids you create and so it’s precisely those people who yeah so that’s one the second for the hairdresser I mean I I thought about this for a while the issue is you know even now you know if you want to solve a plumbing problem you can actually Point your phone and chat GPT at it and you know ask for advice and you can actually get pretty specific instructions and what to do maybe not for all pring problems what that does mean is that there’s a distinction between the knowledge that a skilled Workman work person you know work person has and the actual muscular strength or skills you know to do it so at the very least even those of us who become redundant will I think there will be a huge downward pressure on unskilled wages you can just you know yell out of the window and say hey you you know come turn this for me right because you can sort of see what what has to be done so the disruption I I don’t even think is going to be that segmented and and and if it is segmented it’s the people who are most likely uh you know to to want something to be done done about right so I think I think the elite of production point is a very good one and I think it’s also one to which Britain is is particularly um dangerously exposed because if you look at the structure of British GDP now more than many countries it depends on a set of uh white color services in companies like McKenzie and deoe and whatnot and you know various kinds of things the city accountancy law business consultancy etc etc now I you know I I know now people working in some of these Frontier companies age 30ish who’ve got friends and sometimes housemates with jobs in these companies they earn you know couple of 100,000 quid a year they they’re one of like the top 1% of earners but the frontier a AI people say to me pretty much everything they do is like already automatable and within a few years close to 100% of everything my housemate does is going to be automatable and for sure no one is really prepared for this right and you’ve got a whole set of people who who’ve studied hard they may have gone to accountancy school or whatever they’ve they’ve worked their way up through McKenzie and they’re expecting their life is going to go on a certain trajectory it’s not at all it’s going to be profoundly shocking if if Sam mman drops a model that suddenly basically just automates all of that away very rapidly a lot same’s going to happen with media I mean already happening with the media right I mean um the industry is already dying extremely fast but also um you know GPT 5 or something is going to be indistinguishable it’s going to be able to produce um at least uh the same kind of um uh political media that currently takes up a lot of humans which of course is part of the reason why so much of the political media has become extremely hostile to the tech world just parenthetically um one of the reasons why the Russia gate hoax was spread so much after brexit and Putin in America is because the New York Times tried successfully to pin blame for Trump winning on Facebook as a kind of preemptive move and a lot of the British media joined that hunt because they thought um rather than ask questions about how the New York Times published children his emails far better to try and blame Facebook for it the fact that all of the actual hoax didn’t make any sense to anybody that understood advertising the whole thing was completely bogus didn’t matter so you could see already even a few years ago the old media companies trying to knap the emerging tech companies inevitably these kind of power battles are going to emerge again right because a lot of money and power is at stake the old media companies think correctly people like Ru Murdoch are looking at this nashing their teeth saying we’ve built all of these things and then some 30-year-old kids going to come along and destroy the entire value of these businesses and basically just like completely replace it potentially with New Media companies so that’s going to cause um complete chaos I think on the University point though controversial point to make in this room but we talked before about this about this cycle of of political media academic Elites being repeatedly shocked in the last 10 years so I’ll give an example of something which I think is another one of these things which is coming in the same way if you talk to what are disparagingly known as low voters in Pennsylvania 3 years ago they told you that Biden is obviously scile watch him walking around the stage I had a grandfather like that it’s unarguable right what did the elite um Insider journalists of the New York Times and CNN tell you they said the president is super super sharp in in in in in private so the low information voters pay no attention to the news we’re actually correct and the highly paid experts in ver Commerce were were completely wrong now if you talk if you look at the same groups now group a says there’s far too many people go to universities a lot of people are going to massive debt to get degrees that are basically pointless or and actually even worse than pointless like a waste of their time because they spend three four years at a young age and they get into massive debt um and there’s just no like actual economic rationale for what they’re doing they’ve all been told to do that by the kind of prevailing culture in Elite circles in London um if you voice that sort of attitude everyone laughs at you and thinks thinks you’re an idiot and um if you go to all the think tanks they’ll say no no no we must discuss how we get even more people going to universities and doing even more of these degrees I think that the low information voters are yet again correct and are going to be proved correct and they’re ahead of the curve on the on the AI thing so I think what will happen hopefully what happens with places like this I think is that fewer people will come the quality will go up well we hope um so uh one more aspect so supposing um anthropic succeeds in building this PhD level postdoctoral level you know AI researcher and then proceeds to run a million of them in parallel so Dario says he will produce the next centuries worth of Science in 5 to 10 years uh for those of us who are scientists this means that every copy of Science and nature will have these seminal papers right so we won’t be able to do what we do now which is look at and go H not worth it right that’s just uh and so on each one of them will need to be studied and we won’t be able to you know I mean to actually internalize a breakthrough takes time you have to maybe there’s a new piece of mathematics there’s a whole whole set of phenomenology um so so one is that disruption which is to say I don’t see how you can credibly say that humans sort of understand what’s been learned but then economically what does that mean does that mean that the next 100 Years of patents have now been created and in the current system and become available to this one firm and then all sorts of other firms you know drop in value uh so I mean that also seems like you know information often arrives through the stock market right and so on so so how does that play and and what kind of Demands for so this goes back but so in the case of Education you could say it’s human capital and its value right and now you could talk about you know this kind of uh intellectual Capital know that’s generated so there again how do you see the I mean how do you see the Leading Edge of it will there be uh and then I’ll turn to the question of ownership of these AI companies but let’s maybe just start with do you think this is a plausible worry and under this scenario and and and how might that play out I mean if you if you read the stuff from people like Dario and Sam and Demis and those teams I mean they envisage this being patchy right it does seem to be um like not predicted 5 years ago but it seems possible that these things actually end up being um continue to make rapid progress in in areas like mathematics and physics MH whilst possibly stalling out in other in in other kind of areas so it it looks likely that it’s going to be um in the short term patchy where they suddenly shock and they do amazing things in in certain areas but then possibly don’t make progress in in other things but I think that once they once they get to the point if if they get to the point that Dario talks about in his paper well sorry I I’ll rewind already there is a situation in which um all of the three letter agencies in America people like the nro uh CIA NSA etc etc uh have already begun discussions with all of these companies that this is this has already been happening same thing is happening in Britain with gchq and MI6 and MI5 um the reasons are sort sort of uh quite a lot of them are fairly obvious right it’s to do with what’s going to happen with drones what’s going to happen with cyber offense and cyber defense um what’s going to happen with um the potential for creating new kind of like biological weapons chemical things etc etc so those conversations have already begun also there’s a there’s an obvious conversation which leopolds I say kind of refer to which is really until his essay came out was was not really a very discussed subject in public but it it was this question about security of the labs you you had a situation you’ve had a situation in which on the one hand some of the core entities of the intelligence services are starting to think incredibly hard about what the about how these companies and these products and these models are going to disrupt power and international relations and and War yet if you go to these companies they’re kind of like classic Silicon Valley companies right with basically like no security and everyone bean bags wandering in and out um very good cyber security in lots of ways obviously at places like Google because you don’t want um uh uh you know hackers trivially hacking into Google and they’re obviously extremely good at dealing with things like that but once you start dealing with power you’re into a whole different ball game you’re into a ball game where intelligence Services um run blackmail operations against your family where intelligence Services run honey traps against the head of your security to escape the cyber security defenses that you’ve got there’s a whole bunch of things that state level actors can do to these companies which um is outside the normal Realms of corporate behavior and corporate defenses and all the kind of normal things that people get involved with so I think it’s going to be inevitable uh it all already sort of is happening that um deals emerge between these companies and the Deep state in various ways whereby the Deep state gets access to information about what’s going on information is shared potentially negotiations are for people to be kind of embedded in these organizations so they can see what’s going on and at the same time um the Deep state will provide security of a kind that only states can only people like the FBI the CIA NSA Etc can deal with um a situation in which PLC Intel is is is honey trapping your head of security um and those kind of problems so I think that that that’s already starting to happen um and the connections between the old freewheeling world in which a lot of these companies kind of um you know just had a normal kind of tech life in Silicon Valley in a certain kind of culture that world is rapidly coming to an end for sure whatever else happens that’s those days I think are over right and maybe the economic stuff could generate deeper pressure for State control um exactly it seems almost inevitable that that will happen as well right I mean can you imagine what happened so I’ll give you an example um uh someone I think you you know as well as as I do brilliant physicist called Steve Sue one of his startups um it’s called super focus and it’s working on uh um automate um try customer support things so imagine your say samung or someone I don’t mean that particular company but just is a placeholder imagine dropping all of its like brochures q&as troubleshooting docks into this model you train in model narrowly on all of that and then what how can it do customer support well it turns out that you know vast majority of those kind of things are basically automatable now you look at a country like Philippines something like 10% or more of its entire GDP is basically that industry so you imagine suddenly um you go from a world in which 10% of your country if your economy is operating on something and then technically like the vast majority if not actually literally 100% of it is can be gone like can be gone like that obviously there’s going to be intense political pressure to try and slow that process down as as has happened repeatedly and part of the problem um with this is Mark um there’s a few podcasts that Mark and re one of the kind of guys who helped build the internet and web 30 years ago now a VC guy one of the things he talks about is that whenever this cycle happens historically it’s much easier to imagine the things that will be destroyed and the jobs that will go away than it is to imagine what will replace them and what will come so when you see these revolutions happen you always see everyone warning oh well that’ll be terrible for so and so terrible for so and so toble for so and so and it is and a lot of those predictions are correct but almost no one slash actually no one says but in 20 years time this will be this will exist and this will exist and this will exist so so these political arguments tend to be very asymmetrical with a very clear set of losers um who will Lobby and potentially with a loud voice uh uh for things that are not necessarily in the interest of the broader broader Society right no I think that’s true again it’s the time scales that and read training times that that cause me give me pause with you know the long-term gains which is to say so I’ll ask you one last uh sort of question and then we’ll we’ll open it up to a distinguished audience which no doubt has things to say so um coming back a little bit to uh your uh narrow political experience you know running campaigns uh even you know winning them when uh with with a high percentage how is the practice of politics likely to change over the next 5 years I didn’t get the sense with the American campaign that you know that there was AI based control in the background maybe there were some number of videos produced and that kind of stuff but it seemed ultimately fairly modest but in the face of these advances and General commercial availability of Fairly sophisticated system then presumably the capacity even to pay for on Commercial terms term so what might an election campaign look like 5 years from now um um so I think so I’ll make a general Point first I think a general point is that a lot of these kinds of tools kind of stretch out a phenomenon that already exists which is a very small number of people are just much more capable of building certain kinds of organizations right if you take it to the Limit it’s people like Elon or general Groves there’s not very many of such people and their capacity to build things at speed and scale is um it’s not like a a 5% or 10% or 20% difference it’s like 100x or something kind of difference in capability a lot of these tools kind of stretch that out even further because they mean that people like Elon can do even more even faster whereas the vast major whereas if you compare like take a whole set of tools those tools a small category of people on the power law like Elon can then use as massive Force multipliers but if you give the same tools to the median MP they’re basically as capable as they were before I not very capable so I think it’ll stretch that whole process out even more for sure because that’s just the sort of long-term history of the of these things um I mean one one example I would give is that uh so basically everything that you read about the Russia hoax and me Cambridge analytica Facebook advertising blah blah blah and brexit is complete like just complete nonsense the the interesting thing technically that we did in the referendum was actually use AI stuff for polling and there are possibilities to do polling much better so the guy who did the models for me in 2016 guy called Ben Warner then did models for me in the 2019 election um and they were by far the most accurate predictions um he predicted 364 seats and the result was 365 seats and those kind of um models for seat by seat outcomes are obviously very useful in elections because that you want to know where should your marginal pound be spent where should your effort be spent um what physical locations do you send the candidate to what are the actual crucial issues in the crucial places right so if you can build models that are that are much more accurate of that kind of thing that obviously they they have a lot of value now this guy who who did those for me is now doing a startup and it sounds a bit sci-fi but trust me it actually um works so he had the idea of is it possible to run what you might call synthetic polling and synthetic focus groups inside large language model models and it is possible so for example if he were here now on his laptop if you had a normal political discussion about election and you said well so and so is going to do an advert on blahy blahy blah and they’re going to put it in Pennsylvania and they think that it’s going to persuade uh women between 30 and 50 who are non- colle graduates living in Pennsylvania of this message what do you do okay well you hire some polsters and they go off and then you hire some focus group people and the focus group people recruit people and that takes time and you have to get them in a room and you have to have a discussion guide and you and the normal kind time cycle for this is say a couple of weeks if you do it very very quickly and you’ll spend many many thousands of dollars on it a lot of companies charge like between $5 and $110,000 for a focus group to talk to like six or eight people if Ben were here now he he could type that request into his laptop and say what do women between 30 and 50 non- col graduates in Pennsylvania white or black or whatever think about the following question and it will generate a focus group transcript of synthetic voters and then if you compare that transcript of synthetic voters by that I mean the people don’t exist right the synthetic Creations created by this you know linear algebra inside the inside the model you compare the output of that with an actual focus group transcript of the same people most people can’t tell the difference between the two and you can generate that in 30 seconds so it looks like his idea will work and if it does work then it’ll the whole multi-billion dollar industry for market research generally will be turned upside down but also particularly so specifically your question about election campaigns it will obviously transform election campaigns because as in war speed is critical if you can sit there and go and get a rough answer because you you want R rough answers is all you need in politics you very rarely does it matter like if the right answer is like 58% or 62% right what you’re actually looking for are the very big questions so if you look at the last election for example um it was obvious in quarter 2023 quarter 1 2023 that the the three biggest isues in the US election were cost of living crisis the southern border and the president scile those are the three big things that you’re trying to look for and those are the three big things that ended up defining Define the election so if you’ve got a an automated system that can look for these big things and roughly figure them out and figure these things out fast then obviously it’s going to change um it’s going to change communication generally and political Communication in particular and this assumes that the model is fairly up to date it’s been fed recent information recent newspapers recent now if you combine that with another set of Technologies right for generating video You’re then very quickly in a very very odd world where one person can sit in a room and they can say to system a generate me an advert that persuades the following the the 30 to 50y old women in Pennsylvania of the following proposition up comes a 30 second video you then drop that 30 that 30C video created by one model into your synthetic polling thing and then synthetically pull the AI generated content it seems like a kind of C like if you said it anything like that 10 years ago people would have just thought that’s uh I mean maybe like in 100 years or something that would be possible but that’s basically doable now in crude form like this is not know deployed products and everyone using this but I’ve seen these things happening in front of me and um something roughly this is going to be happening and that’s going to be another if you if you imagine if you consider that the old media now if you pick up the Times newspaper and and look at how it covers polling done in the old way I.E created in the 1940s way it’s coverage is hopeless and incompetent repeatedly right and that’s been around for like 70 years so imagine how completely incapable the old media was going to be of explaining to people what the hell is going on in this new environment one other aspect of running a campaign is as you say to persuade 30 to 50 year olds but on the other hand if you could persuade at the individual level you’d be even better off yeah now a lot of commercial information is in principle good enough right which is to say your phone company knows things about you and and they do sell that kind of information in some aggregated form but I I don’t know for a fact but my understanding is that in fact you could buy a lot of commercial information which could help you understand y much more narrowly not just 30 to 50 but in that particular town or something y so uh so then the the language model can generate you know it’s it’s customized appeals at some incredible rate and then of course you know send off emails and so on so is that in on the way that for sure for sure that’s basically happening now I think I think people doing it are keeping it quite quiet but I’ve talked to people just in the last month who are doing things like this um for sure that that’s going to happen um and again it makes it it makes everything um uh faster and cheaper because because you can um personalize these things instead of having to do instead of printing you know instead of trying to set up say 100,000 generic messages top and tailed to a set of MPS to try and influence them you’ll be able to send you can today send tailor you can drop them into Claude or chpt or something and get those models to basically tail customize them so the MP thinks they’re getting an actual customized letter doesn’t even realize that it’s all being produced by the by the model so for sure that’s going to happen um I mean there obviously there are some restrictions uh so America is less regulated than Britain but there are still some restrictions on how you how you’re allowed to combine data different files like photo files financial data I think in particular is it has various M regul controls on but I don’t know the details of that um but if you think that’s only if you care about obeying the law yes so someone is going to have to think about what if the PRC or Putin decided to do what didn’t happen in 2016 but that now we actually are going to do it they don’t have to care about us law obviously by definition so they could definitely bring together every available data set in one system um in a way that no us campaign could do legally so that’s another thing that they’re going to have to worry about that foreign actors could um what fore could potentially develop things that are actually more able than anything that exists domestically because of your own domestic regulations just one of many thousands of problems coming at the system no so Steve Rosen whose work you know of um you know strategic thinker me he if he’s just giving a talk a couple days back and this point was in the age of precision long range strike when you have peer competitors you know so that’s back to a stalemate at which point exploiting internal division is the thing to try and do and from what you just said that that will the technology for that is here yeah very good so uh I could keep uh come occupied but you’ve all been very patient so uh I think we should open it up I see a hand up can I just ask we are scream online the audience can hear us you just microp app thank you for a really interesting and informative uh discussion um I have a question which is um for for Dominic which is what do you think are the implications and risks of potentially leaving large parts of the World Behind in terms of tech and AI um particularly for example subsaharan Africa and related to this do you have any regrets about supporting Boris Johnson’s uh decision to merge divid and FCO thank you um on the first one it does strike me as likely so there’s already a trend underway of the internet internet kind of bifera at least right into a kind of us version and the PRC version it seems to me almost inevitable that that will continue um uh in coming years and the development of of the AI models will um will exacerbate that I think increasingly that will be a like a core well it sort of already is a question um so China’s been extremely active as I’m sure you know in Africa on this front it’s had people actually deployed there for years on doing things like facial recognition and training models on on African faces no one down there has being protected by gdpr or anything like that um no one even knows really like what the hell the nature of the contracts are between the government of whatever down there and PRC right it it seems so I’m not at all expert on this and I haven’t thought an awful lot about it but it seems likely um to me that increasingly countries will have to choose do you kind of join in on the PRC infrastructure for all these things or do you join in on the um the US infrastructure and very few C countries um possibly like say Russia and India aside are going to have a lot of meaningful choices on on on a lot of these things including the EU um if you you look at the EU uh there are no relevant companies um at the frontier inside the EU legal structure and you also have the EU has passed the EU AI act which whatever one’s view on how to um in in my opinion um uh different views on on the the longer term questions about AI safety the EU Act is kind of the worst of all worlds because it doesn’t actually deal with a set of safety questions um that it purports to yet it also massively kneecaps EU technology industry and there’s already driven lots of um young people to California so in the EU currently says well we’re going to be an independent player in this but you can only be an independent and and it says laughably to me well we’re going to be the global leader in trust in AI but not in the actual technology of AI That’s not going to work that’s obviously doomed so the EU um brought in gdpr with the intention of doing various things to Big tech companies they were told it would have the opposite effect they didn’t listen it had the opposite effect to what they intended it’s now done the eui ACT that’s a disaster the so you think that because of me and brexit I’m I’m partial I would just point out that macron and others have been saying the same thing that I said during the referendum campaign a lot of EU leaders themselves are now saying what vot Le said in 2016 that we can’t go on with the commission regulating like this otherwise we’re going to be turned into a a museum and nothing else so you can see coming down the line just to broaden your point I don’t think it’s just about Sub subar in Africa I think this is going to be a global question um there’s going to be two huge polls Russia will go will go its own way though in lots of ways be Pro be probably like entangled in the PRC thing India could imagine trying to do its own thing as well but almost everyone else is basically going to have a choice of by American or by Chinese Israel is an interesting case Israel is an interesting case you’re right but in lots of ways I think they’ll they’ll be probably they’ll be a hybrid right they’ll buy American and they’ll do their own customized things for certain kinds of security issues and ilas Atco has already set up a lab partly in Tel Aviv so they they have a leading yeah person already Israel of course Israel is is very very unique um and if you talk to the British intelligence services and British Special Forces you also get a very very strong impression of just how amazingly unique Israel is that’s because they’ve got obviously they’ve got a a bunch of extraordinarily smallart people and B they’ve got an actual existential situation in which they’re constantly worried about getting killed and they’ve got a power that might go nuclear and etc etc so they’ve kind of got a set of incentives in Israel to build a whole load of things and to take bill building techn technological capability incredibly seriously um in a way that EU just hasn’t hasn’t felt it at all and you can see this in the response to Ukraine right I mean so I had lots of these arguments in 2020 about drones in the future uh and at that point it was a much more kind of theoretical argument a lot of people in the senior people in the M mod didn’t want to hear it any rationalist particularly in a place like Oxford University would look at the last sit the last couple of years situation with Ukraine and say oh obviously the mod must have changed now right absolutely not in fact the opposite is happening the mod has doubled down on funding all of the things which are completely broken and useless and it is funding that by stealing money from the future and stopping people building stuff with drones and AI in the future exactly as You’ expect from pathological uh pathological mod so Israel does not operate like that I highly recommend this book by Ed lutwak and itan Shamir on Israeli military Innovation it’s really fascinating um good yes please thank you you spoken a lot about like pathological like you just said pathological M You’ spoken a lot about how the fact the British state is almost sclorotic and from what I could tell you don’t think it’s kind of full of very smart people like you said Israel was so I’m just asking what what’s the solution you said you know nothing gets done in White Hall because the treasur and cabinet office control it so what how do you how do you change you know the Civil Service the British government to make it effective so in in a nutshell it the the there are there are lots of incredibly able people in the in in the British system but they tend to be young and what happens is that um Young Smart idealistic people come from places like this and they start in the Civil Service in their early 20s then what happens is a reverse Talent filter kicks in the the most dynamic able entrepreneurial hardworking quit by the time they’re about 40 because they look around at this at that point their ideal their their idealism has been beaten out of them they look around at their bosses and they decide they don’t want to spend the rest of their life like that so they leave the worst kind of e of um of HR type then get promoted to the senior jobs and that’s why in Co you had so many things completely collapsed because you had a bunch of completely useless people as permanent secretaries so the caliber of people at the top of the management of these organizations is imploded over o over a few decades the in my opinion the core disaster was northc travelan and the creation of the permanent civil service and that is the thing that must be destroyed at all costs that’s the thing which is killing us that is the heart of the pathology and I recently came across a fascinating letter from paliston to Queen Victoria in roughly 1830 is and he says in that letter he’s comparing Trends in Europe and England and he says um well obviously in Europe what’s happening is there’s the growth of these permanent bureaucracies that are taking over everywhere um that system could never and will never happen in Britain and if you look at how people like paliston actually ran government departments then they were actually responsible responsibility was real their responsibility to Parliament was real what we have now is fake meritocracy in the Civil Service which actually excludes roughly 100% of the most talented people in the world and completely fake responsibility of ministers to Parliament where they’re actually not in charge of anything and you have fake Cabinet Government overall laid on top of that where the ministers walk down the street and they smile for the cameras and blah blah blah but camera now is cabinet now is completely scripted people don’t believe me when I say scripted they think I must there must be an analogy but I mean it literally so if you’re with the Prime Minister and you have a meeting now this is how a meeting happens you go to Prime Minister study before the meeting and he opens a folder and inside the folder is a document called Chairman’s notes and that is an actual script and and the script says here’s how you read it out to introduce the meeting and then all these different ministers will read out the following things and then at the end you read out this is the conclusion so the conclusion is drafted in advance right so whiteall now operates on the basis of completely literally completely scripted meetings where it’s only a bad meeting if somebody goes off script at which point the official Scurry around afterwards and figure out how to clear it up because people actually discuss the real issue my God like we this is not how meetings work right so you have a whole set of people who are officials and ministers who’ve been trained to live in an environment which is the exact opposite of if you go and had a me a meeting with Jeff Bezos or Elon Musk or with William pit in 1798 the idea sorry yes 398 thought I got my centuries confused there um the idea of pit sitting there and like reading out a script in bullet point fashion and then all of these clowns like reading out their bullet points and then him reading out his I mean would never have happened in a million years but that’s the way that we are actually governed it’s Inseparable from North Kellan North K tralian has to be blown up the same path the reason why what we discussed before about why you see the same problems Across the Western world is this same thing every Western country has allowed these permanent bureaucracies to develop and that’s why we have this exactly the same pathological Behavior everywhere we should return to the Old English system which is real meritocracy and real responsibility to to Parliament rather than the fake meritocracy and the fake responsibility I assume everybody knows about the northcot travelan reforms right not just not just D okay good Joe and then talking about the 1790s I think you there sorry talking about the 1790s so the time when BR there was this called National industrial you know things like the docyard chatam and ports with these huge massive places integrated into into the state with and now we kind of don’t really think that Tech they’re kind of very separate from the state but with El musk kind of being so much part of the new Administration can you foresee a thing where at least in America the there is this sort of national industrial thing where these kind of places that are the tech leading are somehow also integrated in the state or closely integrated into the state the way say the Dos are and maybe Israel is I don’t know yeah yes you you I think you kind of easily imagine that um I mean it’s obviously completely unclear at the moment how the whole experiment has got is going to work out with a section of Silicon Valley Elite deciding that enough is enough essentially like the system is so bad we’re going to have to put aside 30 years of thinking that we just get on with building our companies and ignore those people thousands of miles away the view in Silicon Valley always was that being on the west coast and thousands of miles away from DC was actually a great blessing because we can just do our own stuff and they ignore us and leave us alone but precisely because of the issues that we’ve been talking about these worlds are just inevitably colliding now they neither side can leave each other alone and what happened is the Democrats decided to go for a subet a set of Silicon Valley and Silicon Valley which wanted to be left alone decided okay Two Can Play That Game we’re going to take you out which is what’s happened what happens in government though of course is a very different question um now if you actually have people like Elon go to the White House the center of power and say for example we are not going to try and reform the US Force to make it friendly towards drones we know how the process of these reforms works out everyone just Shar PowerPoints for years to come exactly the same as what happens in the cabinet office when you try and talk about AI it’s completely pointless that’s not how Elon works right um so there’s only basically two ways in which someone like Elon can operate option one is you go into the Air Force and fire the vast majority of the people and restructure it completely very very hard to do in government a lot of things you you can do he could do a Twitter or legal in various ways but the other option is you just say that’s okay we’re going to do it differently all the money that us a force was going to get for drones is no longer going to US Air Force instead we’re going to create the Drone Force it’s a separate legal entity we will Empower it with a different set of laws it will be exempt from all of the normal um uh procurement rules and blah blah blah blah and it will actually build like cic Valley does I.E super fast and optimized for Speed and engineering not optimized for congress all of the insane Dei stuff which they put in parenthetically if you look at what happened on the chips for example Biden tried to put tens of billions into chips less than 1% of that money was spent why because Congress insisted on putting in all kinds of Dei communism into the system to buy off various Democratic um uh pressure groups and trade unions and whatnot Clos brackets so you can like to me that’s like that’s that’s the big question um um not uh you know is there some entity like Doge set up outside the government that gives some advice because that will not really affect very much the core question is um and it affects the deeper questions about the AI staff does a subset of Silicon Valley essentially move to DC and sort of take over at least parts of how the system works and therefore you have this new Elite political Elite emerging and a much more entrepreneurial culture culture and a legal shift in how a lot of these institutions work so if that happens then America will look extremely different in a few years time and the gap between what’s happening in America and what’s happening in Europe will grow even faster but of course the whole point about why I describe these things as pathologically is they act like in as an official said to me Dominic you need to understand you are a virus and we are the immune system it’s our job to expel you from the system as quickly with as little damage as possible and that’s how these permanent bureaucracies see the world and that’s how they will see Elon and the whole Silicon Valley Network that is talking about going there um and those people play you know it’s for it’s for the highest Stakes it’s for power in the most powerful country so um uh a part-time Silicon Valley approach I think won’t work because it won’t be able to break through how those things work but the ability of the people is such that if they actually decide to go for it and they go to Washington and they say we’re going to rip all this up and we’re going to start rebuilding it and they have a white house and president that reinforces that then all sorts of extraordinary things are obviously possible thank you one one more sorry I okay all right well maybe we should switch sides for a minute and then come back yeah hello um Ju Just to confirm that Dom’s story about scripts is true and I i’ I’ve written those scripts myself but it but it’s quite rare for the script not to have been agreed with the minister in advance uh but that depends on the minister yes exactly they’re happy being NPCs yeah U well I wouldn’t necessarily use that language but but the story is completely true um and it still goes on um my my question was slightly different in that um I I I think of this problem as an economic historian and one thing that’s true of industrial capitalism for hundreds of years is that capitalism is internally self- mediating and that if you’re in control of capital it’s hard to realize gains without employing people so the deployment of capital is the combination of Technology people techniques and all kinds of things to try and realize a gain um what AI raises the Spectre of is that you can raise gains without needing other people or needing far fewer people so that implies that you’ll have progressively more concentration of wealth and you’ll have a separation of interests whereby people in control of capital don’t need anybody else to realize upside which is new that’s never happened before historically my concern from a societal level and I think this comes back to your first two questions is like what’s the likely view what’s the benign view um if you have large sede of the population displaced from productive activity they’re no longer contributing and also you have especially in Europe States constructed on the basis of Taxation and most of that taxation comes from income social contributions um fiscal transfers it very little tax actually is derived from Capital itself how do you maintain stable Societies in that sort of future um do you arrive at a situation where people try and exercise political power to prevent these things from happening or uh do you have a simply more redistributive Society where the state gets serious about taxing Capital that that’s the sort of future I think of it’s hard to know which one of those two will happen um but I I I’d be interested in both of your views on that one so I I’ll make a physicist point which you’ve sort of heard me make say along the way you know when I open an economics book and I look at it you know it reminds me of thermodynamics uh which is not shocking because I think well some of the founders were actually people who learned thermodynamics and then you ported the theory over but that’s near equilibrium right so in economics you’re often having arguments about we move this variable and you know the new equilibrium you know and their linear elasticities which you use in some general sense right to to figure it out uh and in physics also if you uh you know if I slowly heat up a glass of water it goes through a succession of near equilibrium states to which thermodynamics applies but if I were to suddenly drop in there you know I mean if it suddenly raiseed the temperature by a million degrees it’s a completely non-equilibrium process it may eventually settle down to something but in between you have a description which is now complet much harder and much more different so again I’m just bringing the set of metaphors I have as a physicist you know to the discussion which is to say I look at these time scales and I go we’re heading for a very non-equilibrium regime and maybe you can tell me that there is an equilibrium State very far away which involves maybe some of what you’re describing but if the process of getting there is vastly quicker than normal equilibrating processes then then I do have to wonder what that you know sort of Journey looks like so that’s sort of the picture in in in in my head I mean you could imagine in principle ways of slowing it down I mean if you thought of harm you know sort of people being laid off as a harm you could economic jargon have a pigovian tax right I mean a carbon tax is supposed to be that or something but as of now I don’t think these are things that anybody’s discussing partly because of the international competitive idea that you know if you don’t get to it then so so all these things are tied together I think in a way that’s very anyway but over to you for maybe yeah so so um couple things point one I think um this agent thing is already happening right one of the most interesting things that’s happening now I think inside these companies is experiments with um kind of agentic companies the concept being that um uh you can that the models can essentially create or possibly not create um they they given some high level goal and then they’re told build a company that can go on the internet and do the following things um and with little other than setting some goals inside open AI or anthropic or whatever the the agentic company then goes onto the internet starts beavering away and builds itself we’ve even seen a weird weird kind of example of this so there are some kind of very sort of fringy experiments that have already happened and sort of leaked out one interesting thing to Google is um a guy set up a kind of crypto chatbot he put it on Twitter the crypto chatbot tweeted at at mark andrees one of the top five um most influential uh Silicon Valley um VCS and it asked andreon for funding out of curiosity Andre publicly on Twitter you can read the whole Twitter exchange why did $50,000 in crypto in Bitcoin or something and this entity then um uh uh this um crypto ball has then then started marketing memes and then created a meme coin currently valued at something like $300 million now who knows maybe that’ll all plumet to zero tomorrow but the point isn’t like what happens to that exact thing it’s just these weird things like are already happening on the internet um and these are just kind of like Fringe little experiments at the moment but inside open Ai and and the big companies they’re experimenting and assuming that that this is going to be a normal thing so at the moment people think oh opening eye is a company shortly opening ey opening eye might be a company with a thousand companies inside it all off doing what doing what you’re talking about um my second Point very quickly is um is the is is just going back to the and recent Point as a as a caution historically we always think we always see how things are going to be destroyed and we never see how things are going to be created mark would say if if you were here every single previous revolution has the same warnings every single time humanity is so creative that people go on and they do new things that’s always happened so far without exception so we shouldn’t just assume that um or at least we should be very cautious about thinking that uh oh these things are just going to destroy all of this activity and humans are going to have no role because that’s not how things are played out before the obvious answer to that is intelligence is intelligence is a quantitative quantitatively different thing therefore history might not play out the same but I’m just putting the the the counterargument the third thing is the point um that that you mentioned the international competition point is critical I think because this is driven a this has been Central to the whole process since kind of Isaac Newton and the Industrial Revolution and everything a few centuries ago right every country has had had this question you know you can read endless things in the 19th century about people talking about the influence of Railways the influence of the telegram this is the devil the Devil is a foot no good will come of it atheism will spread chaos will come you can read letters from people like metan at the time predicting what will happen if all these Dynamics continue if you then fast forwarded him to 1940s Europe with um aitz and whatnot he would say see all of our fears of a century go were completely Vindicated these this automation process these Technologies destroyed the old world they destroyed Christianity they destroyed our old constitutions and um you’ve got indust industrialized Slaughter of millions um we were we were right but no one’s been able to get off the ladder because if you get off the ladder then you’re basically enslaved by your by by other state actors um and if you go back and read vman’s famous essay in 1955 can we survive technology he points to this Dynamic and and say you know there’s not there’s not really any way out of it the theoretical way out of it is um is some kind of one world government and therefore it’s not surprising that some of the Fringe elements of the AI safety movement are essentially now positing a kind of stalinist one world government as the only way actually to control AI which is sort of logical in one sense but raises the obvious question that um we saw how stalinism played out out death on a on unimaginable scale and having a global stalinist government able to restrict and control technology everywhere and anywhere um is uh for sure going to be catastrophic so that’s the fundamental um that I think that’s the kind of a kind of fundamental dilema that no one has a way out of and probably there is no way out of it probant do you have a question yeah yeah you haven’t talked very much about what governments should do uh about uh AI so far so therefore the question if you were special advisor to K starma what would you advise him to do so the first thing I would do is I point out to him how he has destroyed his own capability on AI without even realizing what he done because of how srey and and and the cinet office work uh and I would say to him Step One is obviously to reverse uh the blunder you made on day one without even knowing you’ve made such a blunder uh so you recreate an actual um high performance elite team with these kind of skills around the PM uh what Co showed extraordinarily clearly is the incredible difference it makes between if you have a PM in a normal cabinet room with like 1914 a bun like 20 different uh guys with very similar education pens and paper sitting around who don’t understand graphs and don’t understand technology then you’re going to have disasters um versus what happened when we brought in people that did know these things and the and the extraordinary difference it made to the conversation because the politicians then could actually have actual real experts in the room explaining to them what these graphs meant and and and whatnot so I would say before you get to any kind of serious Poli the the mo the critical thing is you have to have people with technological expertise near the PM but you can only do that if the PM clearly takes it seriously because all of these people can make massive amounts of money and do all kinds of fascinating things or in in companies or be in a place like this which is very conducive and lovely for people to live in no one wants to be no one like that wants to be in a non job around the PM there’s lots of officials who are happy being in non- jobs around the PM but the technical able technological people don’t want to be in that in that thing so you have to have a PM that actually wants takes it seriously a PM that puts his own power and responsibility behind taking it seriously and then has these people there the second thing I would do is I would um I would approach the AI problem slightly obliquely and I would say we’ve just had covid um Across the Western world that was a fiasco we’re going to create the actual entity that should exist to deal with future pandemics because for sure um we are going to face future pandemics and possibly pandemics in which AI plays a role given that these models now are people are experimenting with these models in terms of creating bioweapons and other things and this is like a live issue inside of National Security Council and other parts of um similar sorts of entities in in London so I would say let’s build the actual super duper pandemic entity that should exist but is prevented from existing by current whiteall such an entity inevitably will integrate AI expertise into it and then you will have a set of people who are thinking about possibilities for death on a large scale and War and terrorism and how White Hall works and how power decisions are made at the center of power and how AI models work for prediction and for helping all kinds of other capabilities and you’ll be training a whole set of people on a daily basis to figure out how do how does Ai and and power and death how do we actually manage these things in a live setting so every day these people will be waking up and looking at reports coming in from the Congo or whatever saying the new Cade of blah blah is duh and the model predicts blah and D blah and what do we do about that so you’d have this it very very iterative um process and it’s a sort of thing where you could it’s so important right because you act you could literally save a billion people’s lives it’s a very it’s a great job for a set of people if it’s taken seriously so you could imagine the PM saying okay we’re going to build something but it’s going to be outside all the normal horror show it’s going to be outside white all HR entirely and outside White procurement and outside all of the all the things that make high performance entities IL illegal inside whiteall here’s this Mission it’s going to be part of the PM’s office don’t and if you created that you would then have something which um which I think would be an enormous asset and would be training people to think properly about all of these questions and bringing together theoretical questions and very very practical questions engineering questions and organizational questions that’s my best solution so far to this what about policy policy comes from having PE from people so I think the most important thing you know if you look um if you look at the last few years uh I’ll give you an example of policy the online harms Bill a total Fiasco the online harms bill is there to deal with a much like a a sort of in almost infinitely simple simpler problem than the ones we’ve been dealing about today but if it’s completely Beyond whiteall to deal with that they’ve created an on online harest bill which is completely faral in almost every way so my point is it’s sort of pointless thinking about policy while you have whiteall structured the way it is because it’s just programmed to go inevitably wrong uh I don’t want the current White to regulate or do anything with AI because it’s just absolutely definitely programmed to completely it up you have to think at a more fundamental level the fundamental level is true expertise and actual high performance organizations if you create that then you’re in a position that you’ve got something that could actually generate good policy so there’s one last question there and then we have we’ll move to the reception and you can certainly ask other things but yeah that that person why should it why should we View the government being resistant to AI because they’re resistant to change rather than being resistant to an implementation of AI which will largely to me benefit those disproportionately who are wealthy so I didn’t quite understand the question say again why should we view the government as being resistant to AI because they are resistant to change yeah rather than being resistant to an implementation of AI that seems like it will largely and disproportionately benefits those who already have power and are all three seem to be quite wealthy sort of the Silicon Valley Elites so you’re suggesting that what’s happening at whiteall is that the people in charge are thinking through an approach and their approach is um doing certain things on AI would help billionaires we don’t want to help billionaires therefore we will do what we’re doing is that what you’re suggesting is currently happening um sort of a resistance to an AI which has largely be driven in the interests of those at the top rather than having a more of a holistic view societal wide I think you just you you enormously overrate the amount of thinking that’s that that’s going on about these things I mean if if your model of these guys was correct then they would be acting in various ways to Scupper billionaires with AI right but they’re not I about hesitancy why seem hesitant about figures like Elon Musk and such when they say we want to implement time this sort of way that that your your model of this can’t possibly explain why the PM would nuke his own capabilities inside his own office without even realizing what he’s doing it’s not the most of these things are not um uh if you talk to like the median MP or the median cabinet minister or the median Shadow cabinet minister they’re not thinking about any of these things they’re not it’s not a subject of of interest there and also I mean this question people this room there is also an element of the established Science World and the established AI World which has various power and budgets at the moment and is very keen on telling political people don’t worry this is just not really very fast and this is a lot of Silicon Valley hype and we don’t really need to get them very very concerned so there are also elements inside the kind of traditional scientific World bureaucracy which for their own reasons fundamentally to do with power and money as usual don’t want the government changing things and therefore they are also telling MPS oh a lot of this stuff is overhyped and and you don’t have to engage with it so you’ve got a bunch of MPS who anyway their default program is just wake up see what’s in the Old media and babble about that all day rather than think about sort of questions we’re discussing today and then they’re told by some respected chap from the Royal Society so don’t worry about these Elon mus characters so it’s all you know a lot of California nonsense then of course they can they can do what they’ve been doing for the last X years which is just sort of ignore it and not take not take a lot of it seriously and then get super surprised when certain things happen I think people are worried about El musk figures my point well they’re worried about El musk like figures for sure but not in any sense because they because they have um an accurate model of on and an accurate model of AI and they’re putting these two things together and then coming up with some kind of accurate model of what’s going on in the world they just a lot of policians just have a oh my God the billionaires but a certain kind of billionaires right so if you look at for example the billionaire class in the last election was very very strongly for the Democrats not for the Republicans Democrats raised far more money from billionaires than the Republicans did so it’s which class of which which class es of billionaires is also a critical thing and this goes back to an important I mean very very complicated thing but it’s where the rushia gate hoax about Trump and brexit collides with psychological Dynamics inside the Democrat Party over the last few years which itself generated this back backlash from part of Silicon Valley part of the reason why what’s just happened has happened is because after Russia gate hoax spread a whole set of democrat politicians said very very clearly to Silicon Valley you guys better make sure that all this AI stuff is politically acceptable and by politically acceptable what they meant was espouses AOC like politics in public hence while you see these models over the last few years have all come out they’re basically like AOC parrots as soon as you get them onto the subject of everything to do with sex race history Etc you had the ludicrous thing recently with Google getting making a complete Global spectacle of itself with its image generator thing and making everyone black from 1780s and whatnot um that’s that was to do that was precisely because of behind the scenes uh pressure same kind of pressure and same kind of Dynamics were what led to um embeds from the CIA and FBI and whatnot inside Twitter actually doing live editing and live censorship of things on Twitter at the best of the of of various agencies inside the White House so that whole thing has created this backlash inside Silicon Valley which is a lot of the Democrats essentially sorry it’s connected one other thing which is very important two weeks before the last before the last election John Kerry Presidential nominee for the Democrat Party in 2004 publicly stated on TV the first amendment was a historic mistake and after we win the election we will fix that mistake if I or anybody predicted that such a person would have said this even 5 years ago everyone would have thought that was completely insane couldn’t possibly conceivably happen that Democrat leadership would describe the First Amendment as a historic error but it shows the direction of travel of a large fraction of the Democrat Party and a large fraction of the labor party and a large fraction of Academia these things are all are all connected up and that’s part of the reason why Elon and and another and a kind of counter Elite Network inside Silicon Valley decided to um uh so two years ago I had summer here I was here in summer and I had lunch with my old tutor Robin Lane Fox and he um quoted to me an amazing passage from Cicero where ciso is lamenting the elites of the time and saying that they’re all sitting at their behind behind their wall Gardens tend into their fish ponds and if they keep doing this the Republic will be destroyed I went over to Silicon Valley with this ringing in my mind and I said it to some of these people and said you guys are going to have to get involved and you’re going to have to put aside your fish ponds and lo and behold for those of you can’t see on camera Robin is now clutching his head at the at this thought um uh lo and behold some of these people have put aside their fish ponds and decided to throw their weight in and this is a fundamental reason they that um the democ a lot of these people remember most of these people voted for Obama twice and voted for Hillary a fundamental reason why they switched is because they’ve come to see the Democrat Party in the last four years as a party that sees the first amendment in free speech as a mistake to be corrected and they’ve come to see that as far more dangerous than Trump and that’s part of the motivation of um what an explanation for what’s just happened in America maybe I’ll say one last thing and then we’ll finish which is you know I hope AI turns out to be one of these issues that in fact transcends the usual tendency to polarize along a right left axis just to there’s an interesting case of California it was this bill s SP 1047 for creating liability for very very high-end you know compute being used for things two people who were for liability were Elon Musk and then the company anthropic other people in Silicon Valley were on the other side and did not want that and eventually you know knew some sort of in other words ilon was in fact on the other side from what you might I hope this is more the future in the sense that it’s a species-wide challenge and it would be a shame if it ended up being something that you know was only just ended up becoming another you know there’s the major becomes polarized like everything else it goes like immigration or Di and a lot of these things I think it would be a shame with that uh maybe thank Dominic

FOR EDUCATIONAL AND KNOWLEDGE SHARING PURPOSES ONLY. NOT-FOR-PROFIT. SEE COPYRIGHT DISCLAIMER.