FOR EDUCATIONAL AND KNOWLEDGE SHARING PURPOSES ONLY. NOT-FOR-PROFIT. SEE COPYRIGHT DISCLAIMER.

Understanding the AGI existential threat to humanity as a Multipolar Trap: a Nash equilibrium “Suicide Race” to the bottom. [aka “Moloch” strikes humanity again]

MAX Tegmark discusses with Lex Fridman the AGI race-to-the-bottom as a “Suicide Race” wherein the players are trapped in a Nash equilibrium, also known as the so-called “Moloch” game theory metaphor.

  • MAX. we have the power now for the first time in the history of our species to harness artificial intelligence, to help us really flourish, and help bring out the best in our humanity rather than the worst of it. To help us have really fulfilling experiences that feel truly meaningful. And you and I shouldn’t sit here and dictate the future generations what they will be, let them figure it out. But let’s give them a chance to live, and not foreclose all these possibilities for them, by just messing things up, right?
  • LEX. Well for that, we’ll have to solve the AI safety problem.

In the “moloch” race to the bottom game theory, humans compete against each other in an outcome where everybody ultimately loses. Even though everybody sees the problem and understands the problem, they still can’t get out of the race, or they lose anyway.

Ultimately, the only known human solution to the multipolar trap is sustained and enforced cooperation amongst the parties to the situation.

In the case of the AGI existential threat to humanity, the parties to the situation are all 8 billion people on planet earth, including all people who came before, and all people who may come after today.

Formal Definition: A Nash equilibrium is a set of strategies, one for each player, such that no player has incentive to change his or her strategy given what the other players are doing.

Learn more:

FOR EDUCATIONAL AND KNOWLEDGE SHARING PURPOSES ONLY. NOT-FOR-PROFIT. SEE COPYRIGHT DISCLAIMER.