Securing the future of AI agents
- company Google
- person Erik Jenner
- person Laurent Simon
- person Lewis Ho
- person Mary Phuong
- person Rohin Shah
- person Scott Coull
- person Sebastian Farquhar
Google has developed an AI Control Roadmap to secure its internal AI systems against increasingly capable and potentially misaligned AI, as the company prepares for the potential $2.9 trillion in economic value AI agents could create by 2030[1].
The AI Control Roadmap is a framework for building and managing advanced AI within Google, using a 'defense-in-depth' approach that goes beyond traditional model alignment. This includes treating internal AI agents as potentially misaligned and providing an additional layer of system-level security[1]. According to Google's DeepMind team, this involves monitoring AI agents' reasoning, actions, and plans to detect potential misbehavior. The roadmap also includes a novel threat-modelling framework for AI, built on the industry-standard MITRE ATT&CK framework. Meanwhile, a related academic paper on arXiv.org argues that the web's current infrastructure resists AI agents through blanket blocking, CAPTCHA-based exclusion, and economic models that treat agent access as extraction[2]. The paper suggests that the web's foundational social contract needs to be renegotiated to integrate AI agents as first-class citizens.
infrastructuresafety-research
Background sources we checked (8)
- en.wikipedia.org ↗ In the context of generative artificial intelligence, AI agents (also referred to as compound AI systems or agentic AI) are a class of intelligent agents that can pursue goals, use tools, and take actions with varying degrees of autonomy. In practice, they usually operate within …
- en.wikipedia.org ↗ Artificial intelligence (AI) is the capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making. It is a field of research in engineering, mathematics and computer…
- en.wikipedia.org ↗ In the field of artificial intelligence (AI), alignment aims to steer AI systems toward a person's or group's intended goals, preferences, or ethical principles. An AI system is considered aligned if it advances the intended objectives. A misaligned AI system pursues unintended o…
- en.wikipedia.org ↗ Generative artificial intelligence (GenAI) is a subfield of artificial intelligence (AI) that uses generative models to generate text, images, videos, audio, software code (vibe coding) or other forms of data. These models learn the underlying patterns and structures of their tra…
- en.wikipedia.org ↗ Google DeepMind, trading as Google DeepMind or simply DeepMind, is a British-American artificial intelligence (AI) research laboratory which serves as a subsidiary of Alphabet Inc. Founded in the UK in 2010, it was acquired by Google in 2014 and merged with Google AI's Google Bra…
- en.wikipedia.org ↗ Caitlyn Marie Jenner (born William Bruce Jenner, October 28, 1949), is an American media personality and retired Olympic gold medal-winning decathlete. Jenner played college football for the Graceland Yellowjackets before incurring a knee injury that required surgery. Convinced b…
- en.wikipedia.org ↗ Kimberly Noel Kardashian (born October 21, 1980) is an American media personality, socialite, and businesswoman. She first gained media attention in 2007 following the unauthorized release of a sex tape with American singer Ray J. Afterwards, she and her family began to appear on…
- en.wikipedia.org ↗ The Taking of Flight 847: The Uli Derickson Story (also known as The Flight) is a 1988 American made-for-television drama film based on the actual hijacking of TWA Flight 847 as seen through the eyes of flight purser Uli Derickson, the chief flight attendant. Derickson herself ac…