Compressed Sensing for Capability Localization in Large Language Models

Researchers have identified a method to pinpoint small sets of attention heads in Transformer language models that are essential for specific capabilities, showing that disabling just a handful can sharply degrade performance on targeted tasks. The findings, detailed in a paper by Anna Bair and colleagues, demonstrate that large language models (LLMs) such as Llama and Qwen contain sparse, functionally distinct components necessary for abilities like mathematical reasoning and code generation [1]. The team introduced a compressed sensing-based method that locates these critical attention heads by performing strategic knockouts and a small number of model evaluations [2]. Zeroing out as few as five task-specific heads degraded performance by up to 60% on standard benchmarks measuring the capability of interest, while largely preserving performance on unrelated tasks [2]. The experiments spanned models ranging from 1B to 14B parameters [2]. The results suggest that capability localization is a general organizational principle of Transformer language models, with implications for interpretability, model editing, and AI safety [1]. The work builds on the broader understanding that Transformer architectures, which underpin most modern LLMs, distribute computation across many attention heads whose individual roles have often been opaque. By showing that certain capabilities depend on a small subset of these components, the study offers a pathway toward more targeted interventions in model behavior [2]. The paper was posted on arXiv, an open-access repository for electronic preprints that has hosted over two million articles since its founding in 1991 and now receives about 24,000 submissions per month [10]. The repository is not peer-reviewed, but it serves as a primary distribution channel for research in computer science, physics, and related fields [10]. The authors have released their code on GitHub, allowing other researchers to apply the compressed sensing technique to additional models and capabilities [2]. The compressed sensing approach exploits the inherent sparsity of the functional heads, meaning that only a small fraction of a model’s total attention heads are necessary for any single capability [2]. This sparsity allows the method to identify the relevant heads without exhaustively testing every possible combination, which would be computationally prohibitive for models with billions of parameters. The technique represents a step toward understanding the internal organization of LLMs at a granular level, a challenge that has grown more pressing as models are deployed in high-stakes settings where reliability and transparency are required.