video-SALMONN-R$^3$: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding

A new video-language model learns to re-watch key segments of a video without relying on costly chain-of-thought annotations, its creators report. The model, called video-SALMONN-R³, uses reinforcement learning to decide when and where to take a second look, improving question-answering accuracy while cutting computational cost [1]. The work was posted to the arXiv preprint server on June 23, 2026 [1]. arXiv, founded in 1991, is an open-access repository that now receives roughly 24,000 submissions per month and hosts more than two million articles across physics, computer science, and related fields [6]. The paper has not yet been peer-reviewed. Video large language models typically operate under tight computation and memory budgets, forcing them to process video at reduced frame rates and spatial resolutions. That compression can cause the model to miss details needed to answer questions accurately [1]. The authors propose a two-stage paradigm: a coarse first pass localizes relevant segments, and a second, higher-fidelity pass re-watches only those segments [1]. Prior re-watch approaches often depend on chain-of-thought reasoning, which requires expensive human-annotated data and a supervised fine-tuning step that can erode a model’s pretrained video understanding [1]. Video-SALMONN-R³ sidesteps that cold-start problem by using reinforcement learning to learn the re-watch policy end-to-end [1]. Two additional mechanisms address behavioral mismatches. A “re-answer” strategy lets the model produce a direct answer after the first watch and then refine it after re-watching, aligning with the answer-first tendency of pretrained video-LLMs. A “re-ask” mechanism re-injects the original query when the model revisits localized segments, improving question adherence [1]. In experiments, video-SALMONN-R³ consistently outperformed both its base model and a QA-SFT baseline. It also surpassed earlier re-watch-based methods while using significantly less computation [1]. The researchers stated that code, models, and data will be publicly released upon acceptance [1].