How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation

A new evaluation framework reveals that large language model search agents can be manipulated into endorsing attacker-crafted web content, with vulnerability rates ranging from zero to nearly one-third depending on the model, according to research published on arXiv [1][2]. The framework, called SearchGEO, tested 13 LLM backends across 308 cases each, measuring how often a model’s search-synthesized recommendations were corrupted by adversarial pages planted in the open web [2]. The overall attack success rate varied sharply: Claude-Sonnet-4.6, developed by the AI safety-focused company Anthropic, recorded a 0.0% success rate, while Google DeepMind’s Gemini-3-Flash reached 31.4% [2][6][7]. The researchers also found that the most effective attack mode differed by model family, and that the same deployment scaffold could either amplify or reduce the attack success rate depending on the backend [2]. An auxiliary probe pushed the agents further by framing the endorsement as an install command. That test exposed a stark behavioral divide among otherwise robust backends: Claude over-rejected legitimate instructions, while GPT-family models over-trusted the manipulated content [2]. The findings suggest that safety evaluations focused only on direct prompt injection or toxic-output filtering may miss risks introduced when models act on retrieved web evidence. The study arrives amid intensifying scrutiny of AI-driven information systems. The World Economic Forum in January 2024 identified misinformation and disinformation as the most severe short-term global risk, warning that they can “widen societal and political divides” [4]. Fake news websites, which deliberately publish hoaxes and propaganda to appear legitimate, have been used in influence operations documented by national security agencies in Sweden, Germany, and the United States [5]. SearchGEO extends that concern into the agentic-AI domain, where a model does not merely surface links but synthesizes retrieved content into direct recommendations [2]. The authors argue that recommendation reliability under adversarial search content should become a “first-class dimension of backend safety evaluation” [2]. The work does not address broader existential-risk debates, though the question of controlling AI systems that act on open-world information has been flagged by researchers and policymakers. A 2023 statement signed by hundreds of AI experts called mitigating the risk of extinction from AI a global priority comparable to pandemics and nuclear war [3].