BadWorld: Adversarial Attacks on World Models

A new adversarial framework called BadWorld can force visual world models to produce catastrophic prediction failures using visually indistinguishable input perturbations, according to research submitted to arXiv on 15 June 2026 [1]. The technique exposes what its authors describe as severe structural fragility in models that synthesize interactive video rollouts from a single image. Visual world models, or VWMs, generate action-conditioned future frames from a single context image, a capability increasingly used in robotics and autonomous systems [1][6]. Standard adversarial attacks, which typically rely on comparing outputs against ground-truth future videos, cannot assess VWM vulnerability because attackers lack access to those future frames and cannot predict subsequent user controls [1][2]. BadWorld overcomes both constraints through two technical components: a self-supervised velocity attack that disrupts the model's early denoising dynamics without requiring future supervision, and a trajectory-adaptive bi-level optimization that mines hard control sequences to produce perturbations that remain effective across unpredictable user actions [1][3]. The framework was evaluated on representative VWMs handling both continuous and discrete controls [1]. The resulting adversarial images, which appear visually indistinguishable from clean inputs, reliably triggered incomplete denoising, structural collapse, and control inconsistency in generated rollouts [1][2]. These degradation patterns differ from earlier work on generative world models. A separate study published on arXiv, titled "When World Models Dream Wrong," demonstrated physical-conditioned attacks that induced semantic and logic-level distortion while preserving perceptual fidelity, reducing 3D detection performance by roughly 4% and worsening open-loop planning performance by roughly 20% when attacked videos were used for training [4]. Adversarial machine learning research has historically focused on evasion attacks, data poisoning, and model extraction across classifiers and language models [5]. The BadWorld findings extend this threat landscape to autoregressive world models, where the consequences of structural collapse carry direct implications for safety-critical deployment [1][8]. The AI safety field has gained significant attention since 2023, with the United States and United Kingdom each establishing dedicated AI Safety Institutes, though researchers have expressed concern that safety measures are not keeping pace with rapid capability advances [8]. The authors note that the same attack mechanism also offers a practical tool for privacy protection, as it can drive model outputs into out-of-distribution behavior when sensitive scenes are being processed [1][3]. The paper was submitted on 15 June 2026 to the Computer Vision and Pattern Recognition section of arXiv [1].