OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs

A new streaming framework called OmniMem aims to reduce the memory footprint of audio-visual large language models without sacrificing long-form video understanding, according to a paper submitted on 26 May 2026 [1]. The framework addresses a core bottleneck in long-video inference: the linear growth of video tokens and key-value (KV) caches that strains computational resources [1]. OmniMem introduces a modality-aware memory allocation strategy that manages visual and audio contexts separately, tackling the severe token imbalance between the two modalities [1]. It also employs perturbation-aware memory selection to retain informative and non-redundant KV states, enabling compact memory while preserving long-range understanding [1]. In experiments across three benchmarks — VideoMME Long, LVBench, and LVOmniBench — OmniMem was tested with the video-SALMONN 2+ and Qwen-2.5-Omni models [1]. The framework improved over strong training-free compression baselines by 2-4% absolute accuracy under identical memory budgets [1]. When budget-aware fine-tuning was applied, encouraging the model to consolidate useful information into retained memory, an additional 1-2% accuracy gain was observed [1]. The work lands as the broader research community continues to push the efficiency of multimodal models. A separate preprint from May 2026, indexed on arXiv, explores related code-finding tools for papers, reflecting ongoing efforts to streamline AI research workflows [3]. Another submission from March 2026 similarly catalogs tools for discovering code and data associated with academic papers [4]. While these do not directly address KV-cache compression, they underscore the ecosystem’s focus on making large-model research more accessible and reproducible. OmniMem’s approach differs from prior compression methods that treat all tokens uniformly [1]. By separating audio and visual streams, the framework can allocate memory where it matters most, a design choice that the authors argue is critical for realistic deployment constraints [1]. The paper does not include quoted statements from the researchers, but the abstract details the technical mechanisms and quantitative gains [1]. The submission appears on arXiv under the Computer Science > Artificial Intelligence category [1]. No external funding or institutional affiliations were disclosed in the available preprint metadata.