GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction

A new framework called GRACE aims to improve how multimodal large language models predict viewer sentiment in video advertisements by grounding emotional reasoning in structured, action-centric evidence, according to a paper submitted June 15, 2026 [1]. Standard Multimodal Large Language Models (MLLMs) typically process video through holistic frame representations, which can leave fine-grained, affect-relevant events implicit and complicate precise emotional reasoning [1]. The GRACE framework addresses this by extracting temporally ordered subject-verb-object (SVO) triplets from action-centric video descriptions and grounding the subject and object entities as visual entity crops [2]. The action triplets specify “what happens,” while the grounded visual entity crops anchor “who or what participates in each event” to concrete visual evidence [2]. This structured evidence is then provided to the MLLM to perform clue-enhanced emotional reasoning [1]. Experiments on the Pitts dataset showed consistent improvements over Qwen2.5-VL and Qwen3-VL baselines [1]. The researchers also conducted ablation studies, a cross-dataset evaluation on AdsQA, and transfer experiments on an emotion-focused TVQA subset, which they report further support the effectiveness and generalization of the approach [1]. The work enters a research landscape where grounding reasoning in visual evidence is an active challenge. A separate study introduced Visual Evidence Reward (VER), a reinforcement learning framework that explicitly rewards reasoning traces verifiably grounded in visual content, countering a phenomenon the authors call “visual thinking drift” where chain-of-thought reasoning diverges from visual evidence [4]. Similarly, the Chain of Evidence (CoE) framework decouples perceptual grounding from reasoning efficiency by training models to first identify structured spatio-temporal visual evidence and then perform logical deduction conditioned on that evidence set [5]. GRACE’s approach of augmenting MLLM inputs with explicit event structure and localized visual crops operates on a related principle, aiming to reconnect symbolic event representations to perceptual evidence [3].