ALAS: An Automatic Latent Alignment Score for Audio Language Models

A new metric called ALAS offers a standardized way to measure how well Speech-Language Models internally bind audio frames to text tokens, addressing a gap in evaluating these increasingly common AI systems [1][2]. The Automatic Latent Alignment Score, introduced in a paper posted to the arXiv preprint repository, probes the per-layer hidden states of a large language model to score cross-modal cosine similarity between audio and text representations against a reference derived from the Whisper automatic speech recognition system [1][2]. The metric requires only a frozen forward pass and an off-the-shelf ASR reference, with no training or fitted classifier, and is calibrated to an interpretable uniform baseline comparable across tasks [2]. The researchers applied ALAS to four open-source Speech-LLMs — AF3, Qwen2-Audio, Qwen-Omni, and SALMONN — across emotion recognition on the IEMOCAP dataset, open-ended spoken question answering on LibriSQA, and multi-choice audio understanding on MMAU-speech [1][2]. They found that the depth and strength of alignment reflect each model’s audio-encoder design and the acoustic-versus-semantic demands of the task [2]. The metric tracks but does not duplicate task accuracy, exposing models that score well without genuinely grounding in the audio [1][2]. The work was submitted by Cem Subakan and colleagues, with the initial version posted on May 26, 2025, and a revised version on June 16, 2026 [1]. The paper appears on arXiv, an open-access repository of electronic preprints that, as of late 2024, receives about 24,000 submissions per month and hosts more than two million articles across fields including computer science and electrical engineering [6]. The authors have released ALAS as an open-source library so that practitioners can probe their own Speech-LLMs or apply it to new tasks [1][2].