Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations

A new evaluation framework for sparse autoencoders (SAEs) quantifies how well their internal representations align with human-annotated concepts, addressing a gap in current interpretability research, according to a paper submitted on 23 June 2026 [1][2]. SAEs are neural networks designed to learn efficient, compressed representations of data [3]. In machine learning, such representations are often used for dimensionality reduction, creating lower-dimensional embeddings for other algorithms [3]. The study’s authors note that existing methods for evaluating SAEs in vision models rely on proxy metrics or qualitative inspection, rather than directly measuring semantic correspondence between machine latents and human-defined concepts [1][2]. To address this, the researchers constructed two synthetic benchmarks, synCUB and synCOCO, consisting of paired images that differ in exactly one attribute [1][2]. These datasets enable targeted intervention-style evaluation. The framework introduces Fully-Binary Matching Pursuit (FBMP), a coalition-based matching procedure that supports many-to-one mappings between SAE latents and annotated concepts, and which the paper states consistently outperforms one-to-one baselines [1][2]. For functional validation, the team proposes a Targeted Attribute Perturbation Alignment Score, or TAPAScore, which tests whether matched concepts respond selectively and in the expected direction when image-level attributes are perturbed [1][2]. Under sanity checks, the FBMP matching and TAPAScore were the only evaluated metrics that reliably distinguished trained SAEs from untrained ones [1][2]. Autoencoders, including sparse variants, are part of a broader class of neural networks that learn encoding and decoding functions to transform input data [3]. Convolutional neural networks, which have been the standard for computer vision tasks, learn features through filter optimization and are widely used in image recognition and classification [5]. The new evaluation framework was tested on SAEs trained on embeddings from CLIP and DINOv2, two prominent vision models [1][2]. The findings indicate that increased overcompleteness—where the dictionary of learned features is larger than the input dimension—can reduce perturbation alignment, signaling a decline in interpretability [1][2]. The paper concludes that moderate dictionary sizes offer the best trade-off, producing the most interpretable SAEs [1][2]. Code and datasets for the framework have been made publicly available [1][2].