Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms

A preprint submitted to arXiv in June 2026 introduces a method for auditing large language models by clustering their possible text continuations according to both meaning and the internal computations that produce them, offering a view of model behavior that single-perspective analyses can miss. The paper, titled "Shared Semantics, Divergent Mechanisms," was posted on the arXiv preprint server on 6 June 2026 [1]. It addresses a gap in mechanistic interpretability, a field that seeks to explain the internal computations of neural networks. The authors note that the dominant technique, circuit analysis, is typically "target-conditioned, explaining a single prompt paired with a chosen completion" [2]. This narrow focus, they argue, can hide the full range of ways a model might respond. Large language models, or LLMs, are neural networks trained on vast text corpora to perform tasks such as generation and translation, and are the foundation of modern chatbots [8]. The new approach, termed distribution-level unsupervised feature discovery, works by first sampling many possible continuations for a given prompt. Each continuation is then represented in two ways: by a semantic embedding that captures its meaning, and by an attribution signature that records how the model's internal components contributed to generating that specific sequence [2]. The method clusters these continuations by optimizing a rate-distortion objective that balances semantic coherence, mechanistic consistency, and the number of clusters [2]. The resulting clusters reveal distinct "continuation modes" that are not apparent when looking at semantics or mechanisms alone. The authors report that interventional experiments provide evidence that the cluster signatures correspond to "actionable mechanistic factors" [2]. The work is positioned as a complement to existing circuit analysis and behavioral testing, providing what the authors describe as "a scalable audit of the mechanisms underlying a model's continuation distribution" [2]. The paper appears on arXiv, an open-access repository for electronic preprints in fields including computer science and mathematics that has been operating since 1991 [6]. As of late 2024, the repository was receiving approximately 24,000 new articles per month [6]. The submission is also associated with arXivLabs, a framework launched in 2020 that allows community collaborators to develop and share experimental tools directly on the arXiv website [5]. arXiv has stated that Labs projects must share the repository's values of openness, community, excellence, and user data privacy [5].