LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams

A new artificial intelligence model called LiveStarPro aims to overcome a persistent limitation in video understanding: the inability of current systems to process continuous, hours-long video streams while retaining long-term context and deciding when to respond, according to research published on arXiv [1]. The work addresses what researchers describe as a critical gap in Video Large Language Models, or Video-LLMs. Existing online architectures struggle to simultaneously process continuous video, autonomously determine response timing, and preserve memory over long periods, leading to degraded real-time performance and severe forgetting during extended interactions [1]. The LiveStarPro framework, introduced in a paper submitted June 16, rests on three components designed to tackle each of these problems [1]. The first, Streaming Verification Decoding, or SVeD, is an inference framework that identifies the right moment to respond by using a single-pass perplexity check, removing the need for explicit silence tokens that other systems rely on [1]. The second, Streaming Causal Attention Masks, or SCAM, is a training strategy that forces incremental alignment between video and language across streams of varying length [1]. The third, Tree-Structured Hierarchical Memory, or TSHM, organizes information that would otherwise be discarded into event chains, enabling retrieval from effectively unbounded video streams [1]. To test the system under realistic conditions, the researchers also built OmniStarPro, a benchmark covering 15 real-world scenarios and extending to hour-scale streams to measure long-term recall [1]. In experiments, LiveStarPro outperformed existing methods, recording a 28.9% improvement in semantic correctness and an 18.2% reduction in timing error [1]. The model’s streaming key-value cache also delivered a 1.58x inference speedup compared with the same model running without caching [1]. The challenge of processing continuous, long-duration media is not unique to academic research. The broader video game industry, which generated an estimated $159 billion in global revenue in 2020 across hardware, software, and services, has long grappled with real-time rendering and memory management for interactive streams [3]. Modern games often incorporate livestreaming features, using microphone and webcam inputs for in-game communication, creating a parallel demand for systems that can interpret and respond to live visual feeds [3]. The techniques proposed in LiveStarPro, such as hierarchical memory structures and streaming attention masks, may offer architectural insights for applications beyond academic benchmarks, though the paper focuses on model performance rather than commercial deployment [1] [3]. The model and code have been made publicly available on GitHub [1].