Sam Altman spends much of his time explaining how AI could transform the world for the better and pushing back against doomsayers. Nonetheless, the OpenAI CEO has some fairly unsettling ideas about how AI development could lead to a disaster, or disasters plural. He outlined a couple of these nightmares during an episode of the Figuring Out With Raj Shamani podcast in disturbing detail. The two dystopias stem from almost opposite problems: either humanity losing control of increasingly powerful machines, or one group seizing control of most of the technology. "One is that there’s a loss of control to AI, and we intentionally or accidentally sort of cede control of the future to systems that are unbelievably more capable than we are, and we can’t get it back," Altman said. "And the other is like, one person or company or country gets way too powerful relative to everybody else." The first scenario is less the tired cliché of a robot uprising and more of a soft surrender of humanity’s choices to AI until it’s impossible to take back any agency from the AI software. The other is a more modern, if still clichéd, speculative-fiction scenario: an overwhelmingly powerful technology controlled by one person or corporation, leaving everyone else at their mercy. They are both endlessly discussed by people in the industry and policymakers, but having Altman issue the warning is still notable. Altman, through OpenAI and ChatGPT, is one of, if not the central face of AI to those who have heard the bare minimum about the technology and its flaws. Altman believes those capabilities could bring extraordinary benefits, but his comments suggest he’s not sanguine about the looming chaos that AI carries as well. Handing over decisions Advanced AI taking over because people cede control has only become a real-world concern in recent years as the technology moved beyond simply answering questions and into decision-making territory. AI agents can already interact with websites and complete complex assignments, and developers are working toward systems that require less and less human supervision. The more responsibility those systems receive, the greater the consequences if their objectives diverge from what people actually intended. It’s important to remember that what we call AI now relies on probability prediction in different forms. While very powerful and leveraging huge amounts of data, it can’t actually make judgments like a human, and doesn’t have to be malicious to be harmful. That’s why models tend to opt for the disastrous choice of using nuclear weapons as a strategy when simulating how to run a nation. Humans might not understand or approve of how an AI pursues an objective, even if humans issued the task. And if the AI model has enough power and is embedded in enough critical systems, people might not be able to redirect or stop it…
