FOR EDUCATIONAL AND KNOWLEDGE SHARING PURPOSES ONLY. NOT-FOR-PROFIT. SEE COPYRIGHT DISCLAIMER.

Nick Bostrom Introduction

our current age even in the richest countries you know in the richest start off Society will seem equally impoverished because although maybe some people can buy a lot of stuff there’s a lot of things you can’t buy with any amount of money for example perfect health like the ability to live to 200 years the ability to upgrade your brain to get like you know IQ 400 or the ability to feel happy all the time like you can’t just spend X money and buy happiness right in the future these things would actually be possible that would be technologies that would allow you to achieve those and and that might be very cheap you see so many different permutations of what the future could be that it’s uncertain yeah buckle down so joining us now on open book is Professor Nick Bostrom he is a Oxford professor and the title of the book is deep Utopia life and meaning in a solved world uh Professor first of all it’s great to have you here this is the reason why you have to go to bookstore sir okay I was rumaging through the bookstore I saw your book I said okay this looks interesting I read your book and then I had my assistant uh

Nick Bostrom’s background

reach out to you uh the book is fantastic before we get into the book let’s talk about your background and tell us how this became your passion um well I mean my background is uh I grew up in Sweden and then I’ve been um the founding director of a research institute at Oxford University since 2005 called the future of humanity Institute um and so I was a professor at Oxford until uh like earlier this year when when I resigned and uh I’ve long been interested in the impact of uh transformative Technologies on The Human Condition um my previous book which came out in 2014 was about the future of AI and what could go wrong and what we might need to do to avoid that and then this recent book deep Utopia looks at the other side of the coin uh what happens if things go right yeah so let’s go back that book uh written at least published in 2014 title of that book was super intelligence um I actually uh went to a lecture I I have to confess I didn’t read that book but I went to a lecture at Columbia University when that book came out uh where there was a group of people debating the merits of your book and suggesting in the Elon Musk sort of way that there were catastrophic risks

Comparing Superintelligence vs Deep Utopia

associated with artificial intelligence um this book is the opposite of that the the the the new book deep Utopia so where do you stand in that debate sir I know you’ve written a pro and a con what do you see yeah um it’s not so much that My Views have changed they have always been there both the like the belief that if things go well there could be enormous upside and then also the belief that it’s by no means guaranteed that things will in fact go well it’s just the previous book was focusing more on one side of of that and and now I’m trying to dive more deeply into what where would we actually end up if we imagine suppose we do develop like machine super intelligence they we we solve the safety problem so they don’t all go around a mck and kill humans and we like suppose we get our act together as humans as well so we don’t use these tools to oppress each other or wage war or like to other destructive things like take the best possible case scenario um then then do we end up I think in this technological mature condition where because if we have super intelligence it will help us invent all kinds of other Technologies and then we Face these very profound questions I believe about U the meaning of human life like what would you do all day in a world where like AI could do everything much better than than we could do well I mean there’s a lot there’s a lot there I guess the first thing you know because I’m meeting you for the

Are you an optimist?

first time uh on this podcast uh reading the book I read the book and said that Professor Bostrom is a Optimist on The Human Condition and a long-term Optimist on what you just said we get our stuff together we start to figure things out better as a human race now am I right on that are you an optimist no I mean uh I’ve always struggled with that question because people people ask me and like I’m I’m kind of um attacked from so some people accuse me of being this kind of anti-tech Doomer type and then at the same time other people are hating at me for being this kind of techn optimist transhumanist um I sometimes wish I could just invite all of these different sets of people into the same room and they could like hate on each other directly cut out the middleman um but what are you sir are You The Optimist that I see you as or are you I mean are we I think we don’t know how I mean we we have a lot of uncertainty about how this will we’ve never been through an AI Revolution before um and so we just have to recognize that as far as we know right now um both of these seem in the card um I would I mean if you have to have a label I guess a threatful Optimist maybe okay all right let me let let’s go in the direction for a moment at least if we get things right you you write in

What is post-instrumentality?

the book about entering a period of post instrument ality uh describe what that means to us what is post instrumentality it would be a condition in which um instrumental effort is no longer needed from humans the instrumental effort being the kind of thing where you do something in order to achieve something else so maybe you go to work in order to get a paycheck or maybe you go to the gym in order to be fit and healthy and so a lot of what we do have that uh structure we do X to achieve y um and so at this condition of technological maturity it seems we would enter a post instrumental condition because for almost all outcomes we might seek that would be a shortcut that wouldn’t involve our own effort so this is a more radical conception than what we might call a postwork condition yes a postwork condition would be we wouldn’t have to work to make money because no human labor would be uh economically valuable but that still would leave open the idea that there’s a whole host of other things you need to do to put effort in if you want to be healthy you can’t hire a robot to go to the gym on your behalf in a post-work condition like if if you want to have a good relationship with your kids you have to spend time with them yourself Etc um but the post instrumental condition is more radical for example instead of going to the the gym you could take maybe a pill that would produce the same physiological effects in your body um Okay so uh I mean there so let I’m going to paint two scenarios for you I want you to react to both okay right so the scenario the PO instrumentality

What happens to the world in a post-instrumentalist world?

scenario is one where we become lazier or is it one where we become better or I mean what happens to the universe in that scenario and then there’s the Armageddon scenario where the technology takes over decides that human beings are inferior and figures out a way to eradicate human beings right I mean those are the those are the two scenarios right but the question well I think there are more but yeah certainly like okay there are more you’re right I mean you’re fortunately you’re a way more complex and smarter guy than me I’m just trying to get I’m trying to tease out of you because of your book I’m trying to tease out of you what happens to mankind basically or womankind I just start there so the second scenario is kind of in a sense what the the the previous super intelligence was exploring like the we fail to align the AIS and then they become some antagonistic Force so so that’s the one Ty like so in this book Let’s assume that doesn’t happen we get like you know we survive Prosper there’s material plenty um then you ask would that mean we then all become lazy uh kind of lie on the couch and having robot Butlers that um I think it would depend on our choice ultimately um and our human values we would have the capability if if things go well and we reach this condition of technological maturity to really shape not just the world around us but ourselves as well in whichever way we would want so um I

Will AI make us more productive or make us lazy?

guess I guess what I’m getting I guess I guess what I’m trying to understand is we worked the field uh we we figured out how to irrigate we farmed for ourselves we created these small city states we then industrialized and I don’t know I’m still working super hard Nick I mean feels like you are too I mean you’re a very productive project guy and yet we have all of this technology around us and we we have all of these things that have made us more productive I guess what I’m getting at is let’s say we go to that next level where it’s writing our papers it’s writing our books it’s uh it’s doing our trades it’s doing our stock market transactions it’s doing our medicine does it mean that we unplug or do you think we we we Quantum Leap again into something else more productive well well right now I mean um most of the work I’m doing is because it’s the only way for me to achieve various things that like for example the book I couldn’t just press a button and have the book written I actually had to spend the time writing it myself right and you probably have a whole bunch of goals you’re pursuing that require you actually to invest time and effort and skill now in this scenario where AIS truly achieve a general intelligence and then super human levels of general intelligence um I think um it is much more difficult to think of tasks that we would actually be helpful I I think there are like a few you could look at cases where say consumers have a direct preference that the work be done in a certain way so right now maybe some consumers pay a little bit extra for a trinket that was made by some politically favored group or some indigenous crafts person as opposed to produced in a sweat shop like in Indonesia or something right even if the actual trinket is equivalent we some people care about how it was made so that would be an example where maybe human work would still be needed or or people maybe prefer to watch human athletes compete in the Olympics even if there were robots who could run faster or box harder or whatever right but so with with those carvs though I think it does look like we could have full unemployment full full unemployment right that’s kind of the goal of AI yeah exactly no one needs to work right so but but full unemployment but when you say unemployment in our society people get concerned but in this Society there’s so much abundance and there’s so much wealth it really doesn’t matter you won’t have to work right yeah so um and we see like so our education system now for example to a large extent is structured towards producing uh workers who can be productive and contribute to society so you take kids in you put them at the desk and you’re training them to do as they receive assignments and they are graded and they have to be disciplined and all of that is because right now in the world there are a lot of jobs that need to be done so we need to have people who go into offices and get assignments and Etc now in in in a future scenario where we didn’t need people to do that then I think we would want to like change how we think about education for example like maybe kids could be educated to enjoy life and to develop appreciation for you know the art of conversation and literature and hobbies and appreciating nature and all kinds of other things in instead of like being economically productive okay I mean it’s super super

Are the problems outlined in Deep Utopia coming sooner?

valuable I mean you’re you’re I mean I think this is a fantastic book for so many different on so many different levels because it’s very very thought provoking so let me ask this Nick what what timeline do you give this similar to your last book could we see deep Utopia Pro the problems that you describe coming sooner than perhaps anticipated um yeah I think timelines have been fast since super intelligence came up uh came out uh the last 10 years in AI have been really remarkable and um I see no signs currently of it’s slowing down so I think there is a real chance this might happen in the lifetime of a lot of people alive today um and it will be not just the biggest event in our lifetime but perhaps in the in in the history of human the human species this transition to the machine intelligence era um and it’s remarkable just over the past few years to see how much um the public discourse has changed like when I was writing super intelligence this was a completely neglected topic nobody in Academia it was kind of people snickered that haha science fiction futurism whatever and and now of course we world leaders are are discussing transformative AI all the leading AI Labs have teams specifically researching scalable methods for AI alignment and it’s been like just a radical shift where these ideas that for many years you know I was talking about a few of colleagues and and I think yeah increasingly this just will become a main focus for public conversation as as more people realize what’s what’s about to happen I want you know when I finished reading your book I wanted to ask you this question because but this is a big permutation so we live in this great mystery of course we don’t know our origin we don’t understand what happens to us after our demise uh seems like we’re fairly sensient fairly intelligent uh species but yet we live under this cloud of uncertainty and this Myster uh let’s say that artificial

If AI figured out the meaning of life would it be a good thing?

intelligence was able to resolve that suppose we were able to understand the great mystery of what we’re living in uh the question I have for you this is a little more of an esoteric etherial question does that make our lives more purposeful or less purposeful meaning I sort of feel like the Spectre of death uh forces lots of meaning and lots of intensity in our lives but suppose we had an artificial intelligent situation where we could live forever but even if we didn’t live forever we we understand the great mystery what do you say about all that yeah I mean I think it’s possible it might make our lives less uh purposeful uh after this has happened I think though it makes them perhaps more purposeful right now when we have this immense obligation to try to make sure that this actually pans out right it might be that the human lives we leaving now are much more consequential maybe will shape Humanity’s Destiny for for millions of years to come much more so than any other people who were alive previously in history so it has this paradoxical effect that yeah increasing the purposefulness of current lives if if we do succeed like then you know we’re home and dry and we could maybe spend our time enjoying the future rather than be driven by some very strong purposes yeah but again I I guess that’s the issue so if I lose my purpose or my current trajectory of purpose do I find less meaning in life that’s the I gu guess that’s the big question yeah yeah so that’s that’s exactly the kind of question that the book really dives into and because a lot of the discussion about these issues are has so far been very superficial so people think well what if there were like some increase in the unemployment rate like what would we do with training programs or Ubi or whatever and that kind of people people’s thinking stop there they don’t think through if you actually think through this whole situation where where robots can do not just some tasks but all the tasks and then what happens um so so that’s that’s why I wrote this book and so there’s like yeah really diving into that like about meaning and purpose as well as other values that we have and I think that there are significant challenges to those values in this condition of technological maturity and we will need to I think rethink and maybe drop some of our values that we currently hold there others would survive and kind of re reconsider what what really gives human life meaning at the Fairly fundamental question I’m ultimately positive that there is a good outcome of that like if you go through this process you will it will be different not everything you might value about the current world would still obtain but there will be other values and overall I think it might be such that if people look back on the current time they will just shudder in horror at at what kind of utterly atrocious lives we were leading um um in 2024 by the comparison of of our uh descendants or ourselves if we make it through see it’s interesting we’re we’re doing that already though see we we’re looking back a 100 200 years ago we’re saying oh wow our ancestries there were slaves there was mistreatment different people due to their skin colors a result of which we’re rebuking and condemning people from 200 years ago I guess the question is in 20224 perhaps there will be a very large group of people that rebuke us they’ll say we were spending too much time emitting carbon we were spending whatever it is yeah like I think that you know I don’t know that’s a very good point yeah I think I think it does behoove us like when we look back and and judge all other previous generations we we need to also stop and reflect like how will we look in the eyes of posterity probably we will be appearing pretty rotten as well and like laboring on huge moral errors and stuff um so I think that that is probably um good for wisdom to to pause every once in a while to reflect like at how much we currently take for granted as just everybody believe this clearly must and yet we look at every other generation and we now see that they were wrong or misguided about many things and that’s surely holding true for us but even just setting aside the moral failures of earlier Generations um along with their their heroism in other respects like just say they share grinding poverty that most people experienced 100 years ago or 200 years ago um we wouldn’t want to go back to that but I think our current age um even in the richest countries you know in the richest start off Society will will seem equally um impoverished because although maybe like um some people can buy a lot of stuff there’s a lot of things you can’t buy with any amount of money for example perfect health um like the ability to live to 200 years the ability to upgrade your brain to get like you know IQ 400 or the ability to feel happy all the time like you can’t just spend X money and buy happiness right in the future these things would actually be possible there would be technologies that would allow you to achieve those effects and and that might be very cheap so we we I you know I made this up observation I want to get your reaction to it I feel like there are parts of the world that are becoming less religious uh more secular more atheistic frankly but then at the same time that that’s happening there are other parts of the world that are becoming more fervent like more dogmatic about religion and so uh what why is that in your mind and then if we enter this deep Utopia what happens to religion um yeah I don’t know the answer I think potentially it could have a larger role in this future utopian society in in as much as there would be fewer other things to compete with it it so so right now even if you’re very religious you might still need to spend most of your day just getting by making a living you know tidying up your home like doing all kinds of practical things um if you didn’t have to do all of that you could focus more on what ultimately matters like contemplating the Divine and developing your relationship to to God Etc um now ultimately then it’s it’s a question of whether people would choose maybe people will choose different things there um but I think it’s like one of those values that is likely to survive the transition so many other things like that priding yourself of being a bread winner right would kind of be undermined if if all the bread just is delivered automatically without anybody having to make an effort because robots are doing the baking um but but these more spiritual values would be plausible candidates for what remains in that condition to focus on right unless of course we figure out what the great Mysteries are and then maybe we have more or less religion it’s very hard to know I mean yeah depends on what the answer is right like what what actually is behind the depends on what the answers although I will say this I was in Europe this winter and they were using a church for a conference facility and I thought of my Catholic grandmother I think she would have been horrified that they had you they had repurposed the church for business meetings yeah yeah and and so but did yet there’s other parts of the world where the religious fervor is uh perhaps as intense as it’s ever been so it’s a it’s an interesting time to be alive we are a are a muttly bunch as as a human population what surprised you the most you know after all of your years of focusing on this nick uh when is AI and then Humanity surprised you the most well specifically with AI development is um a surprising thing is how anthropomorphic the current uh Leading Edge AI models are um if you look at chat G pt4 or one of these other systems they really are in many ways very similar to a human mind including even in some of their Foy balls so with some of these systems to get the best performance out of them you almost have to give them a little pep talk in your prompt like encour you’re going to think really carefully about this think step by step this is important try like the idea that an AI actually gives you a better answer because you say those things to it before that would have seemed utterly ridiculous uh 10 years ago right that’s not how computers behave um and yet that’s where we are today so that’s um that’s surprising also like that we apparently have a fairly extended period of time when we have ai systems that are roughly humanlike and practically useful for a wide range of tasks but not yet radical super intelligence it was not clear that there would be an extended period of many years where that was the case you could imagine a different scenario where basically AI did nothing and then somebody in their basement figures out like the key missing thing and suddenly it fums and become super intelligent over a week or something um so those are two and I think this this relatively slow more continuous gradual pace of development in a has also enabled uh more people to start to to realize what is going on so you do see more involvement from from governments and Etc and investors in AI development because it’s not so much foresight but they can kind of see Year bye capabilities increasing and then more people can sort of like connect the lines than than if it were more coming as a bolt from the blue Elam Elam M sometimes teases that we’re living into simulation do you think we’re living into simulation Professor well uh many have asked and few have been I I I wrote I I was the person actually wrote the the original paper on this the simulation argument um back in 2001 or something um I I think it’s like there’s a significant chance of this but I have um refrained from trying to attach particular probability to it yeah well I mean one of the things that would reinforce it is that we’ve got this world that uh you know it’s sort of in complete but man or woman has been left to complete it right I mean you you know we we we didn’t have we have the capability of broadcasting and using radio transmission but thousand years ago or 2,000 years ago obviously we didn’t understand how to deploy it or use it same thing with steel manufacturing or other things so it feels like the world was put together and left for man and woman to complete does that make sense um yeah although that could equally well be the case if we are not in a simulation it seems right so I’m not sure whether that particular property of of The Human Experience tells something like one way or the other with respect to the simulation hypothesis right right well we set all right well we’re at the uh I have at the end of my podcast I’ve come up with my production team five words or phrases I’m going to read out the word then I’m going to ask you to react to you give me a sentence you can give me word or some thoughts okay you ready so we gonna start start with the word future I say the word future Nick what do you say um and I’m supposed to answer with one word or just a phrase no just whatever’s coming into your head you can give me a paragraph if you want um I don’t know hold on for dear life hold on for dear life see see you’re you know you you see so many different permutations of what the future could be that it’s uh it’s uncertain yeah buckle down and uh yeah um yeah okay how about trust I say the word trust what do you think and Rel maybe relatedly Faith um yeah I think um we basically have to have that I think there are more things in heaven and on Earth than are visible Tas in our philosophy um and in particular with respect to these Chang is that we are bringing into the world we are really quite clueless about the bigger picture and we hopefully hopefully we can trust that uh that uh it it all works out for the best but um yeah yeah all right I mean so trust comes with some level of optimism how about I say the word Humanity you say what yeah well work in progress well said I say the word super intelligence you think about what um I I think that there is a lot of Headroom above human cognition um so I don’t think we yeah should be thinking of super intelligence is kind of like a really clever nerdy human who is like a little bit faster at solving the math problem but it’s like really a whole level of like like you comp more comparing humans to like um apes or something like that but perhaps much bigger than that right we’re smart we’re not as smart as we think we are how about deep Utopia I say the words deep Utopia you think what uh hopefully um I I I hope that these problems that the book is wrestling with will become actual problems that we will have to solve in the real world which I think they will if things go reasonably well with respect to these practical challenges we have to face between between now and then very well said um well listen the book was fantastic the uh the title of the book is deep Utopia life and meaning in a solved world it’s by a professor at Oxford University Nick bosom thank you so much for joining us today on open book Thank you Anthony

FOR EDUCATIONAL AND KNOWLEDGE SHARING PURPOSES ONLY. NOT-FOR-PROFIT. SEE COPYRIGHT DISCLAIMER.