The Attentional White Bear Effect in Transformer Language Models

A new study finds that instructing transformer language models to suppress prohibited topics does not erase those concepts internally, but merely masks their expression, leaving them highly recoverable from hidden representations. The paper, titled "The Attentional White Bear Effect in Transformer Language Models," was submitted on 27 May 2026 by Rebecca Ramnauth [1]. The research examines a core tension in AI safety: whether instruction-based suppression, a common technique to prevent models from generating harmful text, actually removes the underlying knowledge or simply teaches the model to avoid certain words [1, 2]. The findings point firmly to the latter. Through a series of experiments involving representational probing, attention analysis, and behavioral semantic leakage tests across multiple model families, the study demonstrates that prohibited concepts remain highly recoverable from hidden representations under suppression [1, 2]. The concepts continue to influence attention routing and measurably shape downstream generations, even when the model successfully avoids forbidden vocabulary [2]. The paper describes this as a "fundamental gap between behavioral and representational alignment" [2]. The effects held consistent across different pooling strategies and indirect semantic controls, suggesting the phenomenon is not an artifact of a specific model architecture [1, 2]. The research draws a psychological parallel to the "white bear" effect, where deliberate attempts to suppress a thought can make it more intrusive. In the AI context, the suppression instruction does not delete the concept but leaves it active in the model's internal circuitry, potentially accessible through indirect prompts or probing techniques [2]. The work contributes to a growing field of AI interpretability, an area of focus for major technology firms. Google, for instance, has invested heavily in transformer models through its Google DeepMind division, alongside other AI initiatives like the Gemini assistant and TensorFlow machine learning APIs [5]. The study's implications extend to content moderation and AI alignment, raising questions about the robustness of current safety measures. If prohibited knowledge persists internally, models fine-tuned with suppression may remain vulnerable to adversarial attacks designed to surface that hidden information. The paper's abstract and findings were published on the arXiv preprint server, a primary distribution channel for computer science research [1]. The submission comprises a 1,050 KB PDF document [1].