Hallucination in World Models is Predictable and Preventable

A new study argues that hallucination in generative world models—where visually convincing rollouts drift from real dynamics—is a predictable data-coverage problem, not an intractable flaw. Researchers introduced a benchmark and techniques to both detect and mitigate the failures. The work, posted to arXiv, identifies three distinct hallucination modes in world models: perceptual, action-marginalized, and scene-diverging. Each mode is anchored to a different stage of the generation pipeline and stems from the model encountering regions of the state-action space it has seen too little of during training [1][2]. A perceptual hallucination reflects a coverage gap in the tokenizer’s reconstruction distribution, an action-marginalized hallucination is a gap in action-conditional transitions, and a scene-diverging hallucination is a gap along the imagined trajectory [3]. To study the phenomenon systematically, the team built MMBench2, a 427-hour dataset spanning 210 tasks with ground-truth actions, rewards, and live simulators. They trained a 350M-parameter world model on it [1][4]. The researchers then developed three lightweight signals that predict where the model will fail, achieving a correlation of roughly 0.80 against rollout change in peak signal-to-noise ratio [2][3]. Two interventions emerged from the analysis. At training time, a coverage-aware sampling technique re-weights data to close coverage gaps. Online, the hallucination predictors serve as curiosity rewards that guide targeted data collection. This second approach allowed the pretrained model to adapt to entirely unseen environments using as few as 50 real environment trajectories [1][4]. The paper concludes that the same signals used to detect hallucination can also drive mitigation, framing the issue as fundamentally one of data coverage rather than model architecture [2][3]. World models are a class of generative system that simulate action-controllable futures, distinct from the large language models that power chatbots such as Microsoft Copilot or OpenAI’s ChatGPT [5][6]. While language models are typically evaluated on reasoning and factual accuracy, world models must maintain physically plausible dynamics over time, making the drift documented in the new study a distinct reliability concern [5].