Event-Grounded Question Answering over Long Audio via Structured Retrieval

A new framework called LA-RAG answers natural-language questions over multi-hour audio recordings by converting sound into timestamped event records stored in a SQL database, according to research published on arXiv [1]. The system retrieves only relevant events before generating responses, sidestepping the context-length limits that constrain current audio-language models [2]. The framework, formally named Long Audio-Retrieval Augmented Generation, uses an open-vocabulary Audio Grounding Model to process continuous audio streams into structured metadata — event names, start times, and confidence scores — which are then inserted into a SQL database for efficient retrieval [4]. At query time, the system resolves natural-language time references, classifies the user’s intent as detection, counting, or summarization, and retrieves only the most relevant events to condition the large language model’s answer [3]. This design anchors responses in timestamped acoustic detections rather than raw audio, a choice the authors argue reduces hallucination on long-duration content [4]. Large language models, or LLMs, are neural networks trained on vast text corpora for generation and analysis tasks, but their reliability can suffer when training data is biased or inaccurate [6]. In the audio domain, existing large audio-language models perform well on short clips yet struggle with multi-hour recordings because of context-window constraints, high query-time cost, and weak temporal localization [2]. LA-RAG addresses those weaknesses by pre-indexing long recordings in an offline grounding mode, enabling low-latency question answering, while also supporting an inference-time grounding mode for shorter, open-ended clips [2]. To evaluate the system, the researchers constructed 24-hour Home-IoT and Industrial-IoT audio benchmarks and augmented CASTELLA, a real-world audio moment retrieval dataset, with question-answer pairs [2]. In offline grounding mode, LA-RAG achieved 76.88% overall accuracy on the Home-IoT benchmark and 71.10% on the Industrial-IoT benchmark, with average query latencies below 0.6 seconds [2]. In inference-time grounding mode, state-of-the-art large audio-language models attained competitive event-detection accuracy on CASTELLA-QA but posted low temporal detection F1 scores [2]. When those same models were augmented with LA-RAG’s structured retrieval metadata, temporal detection F1 improved by 11–17% across baselines while latency also improved [2]. The authors also demonstrated a hybrid edge–cloud deployment in which the audio grounding model runs on-device on IoT-class hardware while the LLM is hosted on a GPU-backed server, enabling low-latency event extraction at the edge and high-quality language reasoning in the cloud [3]. Experiments showed that structured, event-level retrieval significantly outperformed both vanilla Retrieval-Augmented Generation and text-to-SQL approaches [3]. The paper, led by Kartik Hegde, was submitted on 16 February 2026 and last revised on 23 June 2026 [1].