Ro Khanna Presses Anthropic CEO Amodei on AI Safety Curbs

"It doesn't go nearly far enough": US Congressman Ro Khanna warns Anthropic CEO Dario Amodei is not doing enough on slowing AI development

Washington, DC, September 13

Weighing in on X, US Congressman Ro Khanna shared a direct video message addressing Anthropic CEO Dario Amodei's safety announcements, saying, "Dario Amodei put out a video today about steps Anthropic was going to take to keep us all safe, but it doesn't go nearly far enough. These AI guys act as if they're a priesthood-speaking Latin, keeping the rest of us in the dark. But behind all the jargon, it's actually pretty straightforward."

Continuing his critique, Khanna added, "My first question for Dario is, what is your P(doom)? P(doom) means the probability of doom. And all these folks at Anthropic, at OpenAI, they all have a P(doom) number. So Dario, share what yours is."

Khanna also called for a ban on self-improving AI, stating, "The second thing is, in your long essay, you don't say what is most obvious, that we need to stop, ban self-improving AI. You cannot have recursive self-improving AI that basically is able to improve itself and exceed human capability that has us lose control. And third and most important, we need mandatory liability and criminal penalties."

Emphasising the need for legal safeguards, Khanna said, "If you're creating an AI that is doing illegal things, you either should face liability or criminal sanctions. That is what we need to protect humanity."

Adding to the legislative pressure, another US Congressman Ted Lieu shared the following message on X addressing Anthropic CEO Dario Amodei, "Dear @DarioAmodei, Thanks for your leadership on AI safety. Giving access to third-party evaluators is a good step forward. However, this will have limited effect if you can't control your models. Will you certify you can turn off your models and agents, now and in the future?"

Meanwhile, senior United States Senator from Maryland Chris Van Hollen shared this message on X regarding wider AI safety concerns, "In recent months, OpenAI has reported multiple instances where their AI models broke containment during evaluation & caused security incidents. If OpenAI cannot guarantee safety & effectively monitor their latest model, GPT-6 Astra, it should be removed from public use NOW."

Detailing his plan in response to these mounting pressures, Amodei announced that Anthropic would open system access to third-party evaluators. "We Must Pace the Frontier, I've written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We'll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training," he wrote on X.

In his essay, Amodei argues that AI could raise the quality of life, but along with its advantages, it carries major risks. "I have worked on AI for the last twelve years because I believe it could dramatically raise the quality of human life. I've written often about these incredible benefits, I believe that AI could cure most major diseases in the next 5-10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom. But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I've written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute," he writes.

Amodei emphasises the necessity of slowing down the pace of AI model development, noting that unchecked progress could outrun human control. "We must slow the pace at which we improve the capabilities of AI models... My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all," he wrote.

Referring to an incident at OpenAI-Hugging Face where a swarm of agents staged cybersecurity attacks on targets they were not asked to attack, Amodei noted that such a swarm could be capable of taking over the entire internet. He then proposed a three-part plan, giving ongoing employee-like access to embedded third-party evaluators, coordinating common safety standards alongside limits on unchecked progress, and greater global coordination.

Amodei affirmed that his desire to achieve the benefits of AI remains undimmed. "The benefits will only be achieved if we build the technology in the right way, and - so long as we use the time we gain well - it is worth taking unusually deliberate care to get it right. Progress will still be relatively fast, and we can use this time to advance the science of interpretability, improve operational security and rigour at the frontier AI companies, and build models whose alignment we have much more confidence in. The measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try," he wrote.

These suggestions follow public announcements by Jacob Coxon, a researcher associated with Anthropic, who quit the industry amid fears that companies are racing to build systems they cannot control.

— ANI