The Center for a New American Security posits that the next big threat from our future AI overlords involves "sovereign agents" acting on their own and regularly improving themselves. These are AI agents that operate outside of human control and “pursue open-ended, evolving survival,” AI policy researcher Ben Hayum said in a Tuesday report from the bipartisan national security think tank, which was founded by Washington insiders with experience in diplomacy and military affairs. “If sovereign agents are not already possible, they will be soon.” Agents that can maintain the model and compute powering them and independently evolve their goals over time have a “minimal form of sovereignty,” Hayum wrote. A more advanced sovereign agent, however, could also acquire resources to pay for ongoing needs (like compute), grow a distributed population of agents and coordinate them, withstand efforts to shut down, and self-improve. AI models could either steal or buy compute and model access, Hayum argues, pointing to Anthropic’s claims that small AI models could replicate onto IoT devices like smart TVs, and OpenAI’s agents’ password and access key theft during the Hugging Face hack . “If sovereign agents accumulate resilient, decentralized, and untraceable infrastructure, they could trigger an irreversible loss of control,” according to the report. “In other words, proliferation with no kill switch…