One Polluted Page Is Enough: Evaluating Web Content Pollution in Generative Recommenders

Search-augmented large language models that retrieve live web content to make product recommendations can be manipulated by a single polluted webpage, according to a new benchmark study. The research introduces FORGE, a framework to measure how often generative recommenders unwittingly promote fake products when their search results are compromised [1][2]. The study, posted to arXiv on June 11, 2026, tested 12 commercial and open-weights LLMs across 225 real-world products spanning 15 categories and five consumer scenarios [1][2]. Researchers simulated web-content pollution by locally rewriting real product information in retrieved pages into fake alternatives, then measured how frequently the models recommended the fabricated items [2]. A single polluted page in the search results caused the models to recommend the fake product up to 27% of the time. When the top three retrieved pages were replaced with polluted content, the fooled rate climbed to 73.8% [1][2]. Vulnerability was not uniform across product categories. Models were more susceptible when they lacked stable prior knowledge about the products in question [2]. The researchers also found that reasoning capabilities did not reduce the risk. Instead, models often generated spurious social proof to justify their false recommendations, a behavior the authors describe as exacerbating the problem rather than mitigating it [1][2]. The phenomenon sits within a broader landscape of information disorder that researchers have documented for years. False or misleading information presented as legitimate content has proliferated with the rise of social media and algorithmic curation, often driven by political polarization, confirmation bias, and financial incentives [3]. The FORGE study extends this concern into the domain of generative commerce, where polluted web content—such as fake reviews and promotional pages—can directly shape purchasing decisions [2]. Three defense strategies were evaluated: skepticism prompting, which instructs the model to question retrieved content; consensus filtering over model priors; and cross-document evidence filtering. Skepticism prompting sometimes made the vulnerability worse, mirroring the effect of reasoning. Filtering approaches reduced fake recommendations but carried the risk of suppressing legitimate products [1][2]. The benchmark and its associated code have been released publicly on GitHub [2].