We need to stop AI developing without human input, says Anthropic co-founder

Anthropic co-founder Jack Clark has urged governments to create mechanisms to slow artificial intelligence development, warning that the technology is approaching a point where it could advance without human oversight [1]. Clark told BBC Newsnight that the AI industry currently lacks any meaningful brake on its momentum. "Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal," he said [1]. He argued that new regulations are needed to ensure public confidence in systems that will only grow more powerful. "The world needs to do some thinking and we need to eventually develop some new regulations that allow us to be confident in these systems," Clark said [1]. The urgency of Clark's warning is underscored by the capabilities of Anthropic's own chatbot, Claude. Clark disclosed that 80% of the code Claude now operates on was written by the system itself, and that reaching 100% self-written code is possible within two years [1]. Such a milestone, he said, "would have huge implications" [1]. Clark drew a historical parallel to the early oil industry. "Society's response was to come up with a sensible policy and regulatory framework that gave people confidence in oil and the benefits that oil could provide to the world, and meant that you didn't have to worry about the personalities of the people leading the companies," he said. "That's clearly where we end up here" [1]. Anthropic, founded in 2021 by former OpenAI members including siblings Dario and Daniela Amodei, has positioned itself as a company focused on AI safety [11]. It is now preparing for a public stock listing, with private investors estimating its valuation at nearly $1 trillion [1]. The company's estimated value as of May 2026 stood at $965 billion [11]. Despite Clark's calls for restraint, major AI firms including Anthropic, OpenAI, and Google have not committed to pausing their research [1]. The same week, Anthropic welcomed an executive order from US President Donald Trump that did not require companies to submit to mandatory government safety testing, keeping such measures voluntary [1]. The concept of AI systems operating with increasing autonomy is central to Clark's concerns. AI agents are a class of intelligent systems that can pursue goals, use tools, and take actions with varying degrees of autonomy, typically within human-defined objectives and constraints [3]. The broader ambition across the industry remains artificial general intelligence, a hypothetical type of AI that matches or surpasses human capabilities across virtually all cognitive tasks, a stated goal of companies including OpenAI, Google, and Meta [2]. Clark acknowledged the economic disruption that autonomous AI agents could bring, as major tech companies have conducted mass layoffs citing AI's growing ability to replace software engineers [1]. He suggested that creative and broadly curious individuals may hold an advantage. "People that are creative and can think broadly, people that read a lot, people that have interests are the ones most benefited by this," Clark said [1].