Kevin Roose | New York Times, 2023 | Book
“A.I. Poses ‘Risk of Extinction,’ Industry Leaders Warn”
A recent New York Times article reports that the world’s top three artificial intelligence labs (Google DeepMind, OpenAI and Anthropic) are acutely worried about future artificial intelligence advances causing human extinction. These worries are echoed by two of the godfathers of modern machine learning: Geoffrey Hinton and Yoshua Bengio.
Ian Hogarth | Financial Times, 2023 | Article
“We must slow down the race to god-like AI”
A Financial Times article goes into more detail on the shape of the worry: increasingly general and capable systems with unpredictable capabilities and behavior, and the growing possibility that we’ll soon see “god-like” smarter-than-human systems.
Eliezer Yudkowsky | TIME, 2023 | Article
“Pausing AI Developments Isn’t Enough. We Need to Shut it All Down”
A short essay by Yudkowsky arguing for an international treaty prohibiting large AI training runs until the field knows how to make smarter-than-human AI safe—something that Yudkowsky expects to take many decades and a number of major conceptual breakthroughs.
Eliezer Yudkowsky | AI Alignment Forum, 2022 | Article
“AGI Ruin: A List of Lethalities”
A longer essay by Yudkowsky, aimed primarily at industry insiders, describing the many specific obstacles to getting good behavior from powerful AI systems.