Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models

A new explainability framework called LEAF-X aims to make transformer-based automatic speech recognition models more interpretable, according to a paper submitted to arXiv on June 12, 2026 [1]. The method targets a persistent gap between the high accuracy of models like Whisper and the difficulty of understanding their internal decisions [2]. Transformer-based automatic speech recognition (ASR) models such as OpenAI's Whisper are highly accurate, but their predictions remain difficult to interpret [2]. Existing explainable AI (XAI) methods often lack faithfulness and precise temporal grounding, meaning the explanations they provide do not reliably reflect the model's actual computation or align correctly with specific moments in an audio input [2]. Ravi Ranjan Kumar, the author of the paper, proposes Listening with Entropy-guided Attention for Faithful explainability (LEAF-X), a model-intrinsic XAI framework designed specifically for transformer-based ASR [1]. LEAF-X combines three techniques: entropy-guided attention weighting, multi-layer attention rollout, and optional causal ablations [2]. The framework identifies low-entropy, high-impact attention heads and layers, then produces sparse token-to-frame attributions [2]. Unlike perturbation-based explainers, which alter inputs to observe output changes, or raw attention maps, which can be noisy, LEAF-X exploits the internal structure of encoder-decoder and speech-augmented decoder-only models to generate explanations that better reflect model computation [2]. The paper reports quantitative improvements over existing methods. LEAF-X achieved a 32% improvement in faithfulness, 35-39% stronger locality and sparsity, and the most stable attributions among the tested approaches [2]. These metrics suggest the framework provides explanations that are both more accurate and more focused on the relevant portions of the input audio. The work supports the broader push for more transparent and auditable ASR systems, which are increasingly deployed in voice assistants, transcription services, and accessibility tools [2]. The paper was submitted to arXiv on June 12, 2026, and is listed under the Computer Science and Sound categories [1]. The preprint is available in both PDF and experimental HTML formats, with a file size of 11,526 KB [1]. The research community can access the paper through platforms that index arXiv content, including Hugging Face's daily papers page, which aggregates trending machine learning preprints [7]. Hugging Face also allows authors to link their papers to models, datasets, and interactive demos, and supports verified authorship claims through its paper pages system [5].