DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing

Researchers have extended transcoder-based circuit tracing to multimodal diffusion transformers, a step that opens the black box of image-generation models such as FLUX.1[schnell] to detailed causal analysis for the first time [1][2]. The work, submitted to arXiv on 14 June 2026 under the title DifFRACT, trains timestep-conditioned transcoders that faithfully approximate the input-output behavior of MLP sublayers inside the FLUX.1[schnell] architecture [1][2]. By replacing those MLPs with transcoders and linearizing the remaining computation, the authors obtain exact feature-to-feature attribution and recover compact, interpretable circuits [2]. Mechanistic interpretability has advanced rapidly for large language models — systems with many parameters trained on vast text corpora [8] — but multimodal diffusion transformers for image generation have remained comparatively opaque [2]. Existing tools offer only partial views: attention maps expose limited token interactions, while sparse autoencoders can discover interpretable features without revealing how those features are transformed and composed through nonlinear layers [2]. The new circuits reveal mechanisms underlying attribute binding and cross-stream semantic propagation, and they provide causal explanations for systematic generation errors [1][2]. Circuit-guided interventions are substantially more precise and effective than standard SAE-based steering, according to the paper [2]. Empirically, the transcoders match or slightly outperform sparse autoencoders on the sparsity-faithfulness tradeoff [2]. The preprint appears on arXiv, the open-access repository that has hosted more than two million e-prints since its launch in 1991 and now receives roughly 24,000 submissions per month [6]. The paper’s code is available on GitHub under the DifFRACT repository [1][2].