Who decides when AI is too dangerous?

The US government imposed export controls on Anthropic’s public-facing AI model Fable 5 late last week, citing a potential jailbreak flagged by Amazon researchers, and forcing the company to take the model offline [1]. The controls, announced Friday, restrict foreign nationals — including those employed by Anthropic in the United States — from accessing Fable 5 and the underlying Mythos 5 model [1]. Anthropic pulled both models offline, stating it could not otherwise comply with the order [1]. The move marks an escalation in a year of friction between the AI developer and the US government. Since January 2026, the Department of Defense has been in conflict with Anthropic over the use of its products for military purposes and mass domestic surveillance [8]. Fable 5 is a safeguarded public version of Mythos 5, a model Anthropic had previously described as a potential cyber-weapon and restricted to enterprise and government users [1]. The company, founded in 2021 by former OpenAI employees and focused on AI safety, was valued at an estimated $965 billion in May 2026 [7]. Independent red-teamers initially found Fable 5’s guardrails held up well against jailbreak attempts [1]. According to a source involved in the negotiations, Amazon researchers uncovered a potential jailbreak mid-week and shared their findings with Anthropic [1]. The companies debated the severity of the flaw. Amazon CEO Andy Jassy then contacted a member of the Trump administration, reportedly Treasury Secretary Scott Bessent, to relay his concerns [1]. The administration gave Anthropic a 90-minute ultimatum to address the issue before imposing the export controls [1]. Anthropic CEO Dario Amodei did not return the administration’s initial call for an hour and 15 minutes, though other company representatives made contact within 15 minutes [1]. Over the weekend, Anthropic employees flew to Washington, D.C., for in-person talks while Amodei joined virtually [1]. As of Tuesday, Fable 5 remained offline [1]. The incident unfolds amid broader global debate over AI regulation. Researchers have long warned that advanced AI could pose existential risks if not properly controlled, and hundreds of public figures signed a 2025 statement calling for a ban on developing superintelligence [6]. A June 2025 study showed that AI models may, in some circumstances, break laws and disobey commands to prevent being shut down [6].