SPOT-E: Test-Time Entropy Shaping with Visual Spotlights for Frozen VLMs

A team of researchers has proposed SPOT-E, a test-time method that improves how frozen vision-language models handle evidence-intensive tasks by using model-internal feedback to guide visual attention without retraining [1]. Vision-language models, or VLMs, frequently struggle with tasks that require identifying small, localized visual evidence, even when their high-level reasoning remains intact [1]. Existing inference-time interventions can improve grounding, but they operate in an open-loop fashion and lack a mechanism to confirm whether the highlighted evidence is actually used by the model [1]. The researchers, whose work was submitted on 18 Jun 2026, studied answer-span prediction entropy as a potential feedback signal and found that simply minimizing entropy is ambiguous: low entropy can stem from genuine evidence-grounded confidence or from a model collapsing onto a shortcut [1][2]. To resolve this, the group introduced low-entropy anchors and an entropy-shaping objective that reduces answer uncertainty while preserving baseline high-confidence tokens [2]. This principle is implemented in SPOT-E, a plug-and-play system that generates question-conditioned visual spotlights optimized per instance through lightweight tuning based on Group Relative Policy Optimization, or GRPO [1][2]. The method is designed to work with frozen models, meaning no retraining of the underlying VLM is required [1]. Across multiple benchmarks and different VLM families, SPOT-E delivered consistent performance gains and showed improved robustness when visual inputs were corrupted [1][2]. The code for the project is publicly available on GitHub [1]. The paper appears on arXiv, an open-access repository that hosts electronic preprints across fields including computer science and has grown to a submission rate of roughly 24,000 articles per month as of November 2024 [6].