While the term evolution is usually applied only to biological systems, it can be viewed more generally: chemical, biological, technological, etc. On the whole, evolution has consisted of a sequence of feedback cycles that are getting faster and faster over time. We are about to enter another phase of evolution, based on digital intelligence rather than human minds. AI is a faster feedback loop. It can run a lot faster than people can think, but it is not fully general yet. As AI develops into a useful tool in more and more disciplines, our civilization will reduce its dependency on human intellectual labor. This is another way of saying that we may slowly lose control. The risks of AI development are numerous, including misuse risks, misgeneralization, and loss of control. However, the potential benefits are also immense. This is why we need to start working together to coordinate AI development. It only takes one AI project gone wrong to endanger everybody. AI is not like other technologies. It can and will overwhelm us if we are not careful.
0:00 Intro 0:22 Contents 0:30 Part 1: From H2O to singularity 0:55 Predicting an exponential future is hard 1:08 Underestimating the rate of progress 1:38 Why technology improves exponentially 1:49 Speed of feedback cycles 1:53 Example: speed of feedback in learning chess 2:38 Example: playing chess against an AI 3:03 Getting short feedback cycles is essential 3:24 Cycle 1: chemical evolution on early Earth 4:02 Cycle 2: biological evolution in individuals 4:53 Cycle 3: language and societal evolution 5:23 Cycle 4: technological evolution 5:59 Cycle 5: intellectual evolution and the singularity 6:38 Evolution needs feedback 7:07 Part 2: AI is a faster feedback loop 7:11 AI is not like other technologies 7:29 AI can even improve AI technology itself 7:47 Example: humans out of the loop in high frequency trading 8:12 How human jobs will change 8:46 With AI, jobs will become obsolete faster than replacement rate 9:22 Humans stepping back from previous types of work 10:03 Example: writing a program in unfamiliar domain 10:29 Abdicating control over our destiny? 10:53 Part 3: A momentous moment 11:01 The beginning of artificial life 11:27 Is this amazing, or terrible? 11:44 Positive: Unlocks utopian science fiction tech 12:29 Negative: Many risks of AI development as well 13:20 Communicating these risks is hard 13:42 We need coordination in politics 14:07 AI can and will overwhelm us if we don’t pay attention 14:38 Different from nuclear risk: child analogy 15:04 Conclusion 15:51 Outro