Does AI Reviewer See the Full Picture? Attacking and Defending Multimodal Peer Review

A new benchmark called PaperGuard has been introduced to systematically evaluate and defend AI-generated scientific peer review against domain-specific, cross-modal attacks, addressing a gap where current robustness studies are overwhelmingly text-only [1][2]. The framework arrives as Large Language Models (LLMs) and Multimodal LLMs (MLLMs) are increasingly integrated into scientific peer-review workflows, a development that introduces novel risks for adversarial manipulation [1][2]. Scientific papers are inherently multimodal, with figures conveying core evidence alongside text, yet existing defenses have not accounted for this complexity [2]. The problem is distinct from standard jailbreaking, as a peer-review attack seeks to induce a targeted failure, such as inflating a review score, rather than a general safety policy violation [2]. No practical defenses for this specific threat existed prior to this work [2]. PaperGuard is built on three pillars. The first is a new multimodal peer-review dataset spanning multiple scientific domains [1][2]. The second is a unified suite of attacks, including black-box prompt injections and white-box perturbations, specifically designed to target both text, using the Greedy Coordinate Gradient (GCG) method, and figures, using Projected Gradient Descent (PGD) [2]. The third pillar is a practical defense motivated by the long-context challenge of academic papers; it uses chunk-based embedding search to efficiently localize and mitigate harmful instructions [2]. Extensive experiments conducted across state-of-the-art models confirmed that AI reviewers are pervasively vulnerable to these attacks [1][2]. The research establishes foundational protocols and an actionable defense intended to pioneer trustworthy, attack-resilient AI-assisted scholarly reviewing [2]. The work was submitted on 10 June 2026 and is hosted on arXiv under the Computation and Language category [1].