P(doom) Roundup. What probability do (super-smart) people put on AI killing everyone?
P(doom) Roundup
P(doom) is the estimated percent probability of AI killing everyone. In fact, nobody knows. The future of AI is extremely uncertain. sources
- Amodei: 10-25% (Dario, CEO and co-founder of Anthropic, was former leader of long-term safety research at OpenAI)
- Buterin: 10% (Vitalik is the inventor of Ethereum)
- Christiano: 50% (Paul is Former Head of Alignment at OpenAI, innovator of RLHF, reinforcement learning from human feedback)
- Hendrycks: 80% (recently updated from 20%, Dan is the founder of, Center for AI Safety (CAIS) and author: Natural Selection Favors AIs over Humans)
- Hinton: 10% (Geoff is Professor Emeritus, Univ. of Toronto, one of three so-called “Godfathers of AI”)
- Karnofsky: 50% (Holden is Co-founder and Director of AI Strategy of Open Philanthropy)
- Khan: 15% (Lina, Chair of the FTC, a self-proclaimed optimist on the subject)
- Leike: 10-90% (Jan is Head of Alignment, OpenAI)
- Mowshowitz: 60% (Zvi Mowshowitz is outspoken on the topic)
- Musk: 20-30% (Elon is a technologist and founder of SpaceX and Tesla, and private owner of X formally known as Twitter)
- Roose: 5% (Kevin is a leading AI journalist for The New York Times)
- Shapira: 10-90% (Liron is a tech entrepreneur – see his explanation below)
- Soares: >95% (Nate is Executive Director of the Machine Intelligence Research Institute – MIRI)
- Yudkowsky: >95% (Eliezer is the Founder of the Machine Intelligence Research Institute – MIRI and LESSWRONG)
More surveys…
- 9% chance of catastrophe, 1% chance of extinction (Extinction tournament, AI non-experts, median)
- 20% chance of catastrophe, 6% chance of extinction (Extinction tournament, AI experts, median)
- 10% (Average ML researcher, early 2022, before ChatGPT)
- 26% (Average American citizen)
- 30% (Average AI alignment researcher)
- 40% (Oct 2023, Average AI engineer)
- 50% (BACA Research at the British Association of Clinical Anatomists)
- 80% (AI researchers at Conjecture)
Consider four (4) scientific facts: (1) for 3.5 billion of years natural selection has been the driving force behind the evolution of all life on earth. (2) Human intelligence, being superior to all other animals, has allowed humanity to dominate and control our planet. (3) It is a fact that natural selection favors intelligence. (4) Humans are now creating a super-intelligent machines which are currently uncontrollable, uncontainable and un-understandable – AIs are considered a “black box” technology.
New to the P(doom) problem? The people in this video clearly explain the problem… Don’t Look Up – The Documentary: The Case For AI As An Existential Threat
For example, Liron Shapira has worked on the problem for 10 years:
“I think there’s like a 50 chance that I’m gonna die for AI related causes by 2040… When I say 50 I really mean like 10 to 90… to make decisions on just the fact that it’s probably at least 10 is scary. “
My P(doom) is 50%.https://t.co/E3TIosooKW
— Liron Shapira (@liron) November 29, 2023