Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies

Researchers have introduced Symbolic Mechanistic Data Attribution (SMDA), a framework designed to trace how individual training examples shape the high-level behavioral policies a language model learns, bridging a gap left by existing data attribution methods [1][2]. The framework, detailed in a paper submitted to arXiv on 28 June 2026, fits a closed-form Ridge regression over sparse autoencoder (SAE) features to model a target behavior. It then analytically decomposes how each supervised fine-tuning (SFT) example shifts that policy through two pathways: feature-activation changes (Delta_X) and output-probability changes (Delta_Y) [1][2]. The authors applied SMDA to distill a symbolic policy for refusal behavior in Llama-3.2-3B-Instruct, analyzing 200 SFT training pairs [1][2]. The analysis surfaced systematic safety gaps. The symbolic policy's coefficients revealed deficiencies in the base model's handling of categories such as religious stereotyping [1][2]. The per-feature decomposition also showed that harmful and harmless training pairs exert qualitatively different influences on certain features, offering a mechanistic explanation for their divergent effects [1][2]. A further finding was that individual training pairs routinely exhibit cross-feature interference. This allows SMDA to flag examples whose dominant effect lands on unintended features, a capability the authors argue makes the tool more fine-grained than black-box influence functions and more scalable than manual circuit analysis [1][2]. The work lands as the broader research community continues to build infrastructure for model interpretability and reproducibility. arXiv, where the paper is hosted, has integrated with Hugging Face Spaces through its arXivLabs program to embed interactive demos directly alongside papers, aiming to let a wider audience explore model behavior without writing code [4][5]. Hugging Face Spaces has hosted over 12,000 open-source machine learning demos since its launch in October 2021 [4]. Large language models like Llama-3.2-3B-Instruct are a class of machine learning model with many parameters, trained with self-supervised learning on vast text corpora [8]. The SMDA framework targets the supervised fine-tuning stage, where models are adapted to follow specific behavioral policies. While the paper focuses on refusal behavior, the authors present SMDA as a general diagnostic tool for combining mechanistic interpretability with data attribution [1][2].