M$^3$Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions

Researchers have introduced M³Exam, a new benchmark designed to test how well language agents handle multimodal memory in realistic user-agent interactions, moving beyond the sparse visuals and straightforward content of earlier evaluations [1]. The benchmark, detailed in a paper submitted to arXiv on 5 June 2026, targets a gap in existing assessments, which the authors say assume human-to-human exchanges and fail to evaluate reasoning over authentic multimodal file interactions or the interpretation of concealed user information [1]. M³Exam is built around query-centric multimodal conversational memory and includes 239 multi-session conversations across 15 persona scenarios, comprising 3,025 dialogue rounds with 1,799 multimodal artifacts, paired with 5,150 evaluation questions [3]. Beyond standard retrieval and multi-hop questions, the benchmark introduces two question types absent from prior work: Thematic Reasoning, which demands domain knowledge implicit in a user’s context, and Implicit Inference, where the answer relies on information the history implies but never states outright [3]. Testing the latest Multimodal Large Language Models and frontier agent memory systems on M³Exam revealed persistent gaps in cross-modal grounding, cross-session reasoning, and the efficiency cost of accumulating multimodal context [1]. To address these shortcomings, the team also proposes M³Proctor, a multimodal memory method that detects query modality bias and consumes raw visual sources only on demand [1]. The approach improved accuracy by 13% while cutting index-construction time and retrieved tokens by over 70% [1]. M³Proctor attained the best overall score among agentic-memory systems at 0.484, compared with 0.456 for the previous best, with the largest gains on the benchmark’s hardest dimensions [3]. The work arrives as language agents are increasingly deployed over accumulating multimodal information [8]. A separate benchmark, M³Eval, has also recently been proposed to probe memory capabilities in multimodal models through video-based question-answering tasks grounded in cognitive psychology, examining retention, interference, integration, and symbolic memory [9]. Code and data for M³Exam have been released at an anonymous repository [3].