How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation

A new mechanistic study of large language models finds that the inline citations used to verify answers in retrieval-augmented generation may not reflect genuine source use, revealing a disconnect between a model’s apparent reasoning and its internal computations [1]. The paper, submitted in 2026, provides what its authors describe as the first mechanistic account of how an LLM decides to attach a citation while answering a factoid question [1]. Retrieval-Augmented Generation, or RAG, is a widely adopted technique designed to ground model outputs in external documents, often by appending citations that users can check [1]. Large language models are neural networks trained on vast text corpora for tasks such as generation and summarization, but biased or inaccurate training data can undermine their reliability [2]. RAG was developed in part to address that fragility by tethering claims to retrieved sources. Using the Llama-3.1-8B-Instruct model in a controlled setting built on the PopQA dataset, the researchers applied an activation patching approach to trace the computational pathway responsible for citation [1]. They found that the mechanism is not a single localized component but a distributed, multi-stage “attributional ensemble” of attention heads and MLP layers [1]. When the team amplified or attenuated only those critical heads and MLPs, they repaired over 90% of missed citations and eliminated 69% of spurious ones on PopQA, all without harming answer accuracy [1]. On the multi-document HotpotQA benchmark the gains were more modest, but the same component set still pushed citation rates in the intended direction, suggesting the underlying mechanism is not dataset-specific [1]. The results indicate that inline citations can create a false sense of security, because the model’s observable citation behavior may diverge from the internal pathway that actually produced the answer [1]. Benchmark evaluations for LLMs routinely attempt to measure reasoning, factual accuracy, and alignment, yet this study underscores how surface-level metrics—such as the presence of a citation—can mask deeper unfaithfulness [2]. The work was conducted with support from arXivLabs, a framework that lets collaborators develop and share new features on the arXiv platform while adhering to the site’s values of openness, community, and user data privacy [1].