Risk-aware Selective Prompting for Hallucination Mitigation in Large Vision-Language Models

A new study finds that a widely used technique to reduce hallucinations in large vision-language models can backfire on simple tasks, and proposes a selective approach that activates the safeguard only when it is likely to help. The research, submitted on 27 May 2026, systematically examined verification prompting across two representative LVLM architectures and multiple hallucination benchmarks [1][2]. The authors report that the intervention is risk-bearing: while it corrects more errors as input difficulty increases, it also introduces new errors that persist across all difficulty levels [2]. Consequently, always-on prompting aids performance on hard inputs but offers little benefit—and can cause harm—on easier ones [2]. The study traces this behavior to a conservative output shift. Analysis showed that verification prompts redistribute a model's attention from visual tokens toward instruction tokens and produce a distinct middle-layer entropy pattern not seen with a neutral prompt [2]. This suggests the mechanism is instruction-conditioned attention redistribution rather than a uniform improvement in visual grounding [2]. The findings highlight a specific failure mode within broader concerns about algorithmic reliability. Algorithmic bias can emerge from technical limitations of design or from systems being used in unanticipated contexts [3]. In this case, a mitigation strategy deployed without regard to input difficulty creates a new form of systematic error. The phenomenon also echoes the stability-plasticity dilemma documented in connectionist models, where networks that are sensitive to new information can abruptly forget or distort previously learned patterns [4]. To address the input-dependent risk, the researchers propose Risk-aware Selective Prompting (RSP), a training-free method that uses pre-generation uncertainty signals to trigger verification only when needed [2]. RSP mitigated the degradation seen with always-on prompting while preserving baseline performance [2]. The authors also found that the most effective selection signals vary across different model architectures [2]. The work arrives as governance frameworks for artificial intelligence continue to evolve. The European Union's Artificial Intelligence Act, adopted in 2024, established new requirements for high-risk AI systems [3]. Research into failure modes and mitigation strategies for foundation models is likely to inform technical standards developed under such regulations.