FutureOmni: Evaluating Future Forecasting from Omni-Modal Context for Multimodal LLMs

A new benchmark called FutureOmni has been introduced to test how well multimodal large language models can forecast future events using both audio and visual cues, a capability that has remained largely unexamined in existing AI evaluations [1][2]. The benchmark, detailed in a paper submitted to arXiv on January 20, 2026, is described as the first designed specifically for omni-modal future forecasting from audio-visual environments [1][2]. It was constructed using a scalable, LLM-assisted pipeline with human oversight and comprises 919 videos and 1,034 multiple-choice question-answer pairs spanning eight primary domains [1][2]. The task requires models to perform cross-modal causal and temporal reasoning and to leverage internal knowledge to anticipate what happens next [1][2]. Evaluations were conducted on 13 omni-modal models and seven video-only models [1][2]. The results showed that current systems have significant difficulty with audio-visual future prediction, especially in scenarios heavy with speech [1][2]. The highest accuracy recorded was 64.8 percent, achieved by Gemini 3 Flash [1][2]. To address these performance gaps, the researchers curated a 7,000-sample instruction-tuning dataset and proposed a training approach called the Omni-Modal Future Forecasting strategy, or OFF [1][2]. Subsequent tests on the FutureOmni benchmark and other established audio-visual and video-only benchmarks indicated that the OFF strategy improves both future forecasting ability and model generalization [1][2]. The paper's authors, which include Jinlan Fu, have publicly released the project's code and datasets [1][2]. The code is available on GitHub, and the datasets are hosted on Hugging Face, a platform that has collaborated with arXiv since 2022 to make machine learning research more accessible by embedding interactive demos directly alongside papers [2][3][4][5]. This integration allows users to find open-source demos on a paper's arXiv page and test models without writing any code [3][5]. The development of FutureOmni comes as the field of large language models continues to advance rapidly. Large language models are machine learning models with many parameters, trained on vast amounts of text for tasks like language generation [7]. Companies such as DeepSeek, a Chinese AI firm founded in 2023, have recently drawn attention by producing models with performance comparable to leading systems but at a fraction of the reported training cost [6]. DeepSeek's R1 model, for example, was trained for a reported $6 million, far less than the estimated $100 million cost for OpenAI's GPT-4 [6]. The FutureOmni benchmark adds a new dimension to these evaluations by focusing specifically on the predictive, rather than retrospective, capabilities of multimodal systems [1][2].