Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Models

New research reveals a sharp geometric divide between what large language models can detect and what they can control, challenging a core assumption of AI interpretability. A team found that while models perfectly identify hallucinated content, the internal directions for detection and intervention are nearly orthogonal. The study, posted to arXiv on June 23, 2026, tests the premise that knowing where a behavior is represented in a model's activations allows researchers to modify it [1]. On a Gemma 2-2B-it model, the researchers examined two behaviors: output formatting and hallucination. For formatting—clean JSON versus markdown fencing—the direction that detects the behavior and the direction that controls it collapse onto a single axis [2]. Hallucination tells a different story. The model detects fake entities with perfect linear separability, achieving an AUC of 1.000 from layer 5, yet the detection direction sits at a cosine similarity of just 0.12—an angle of roughly 83 degrees—from the direction that produces a refusal [2]. A detector built from activations, without chosen tokens, similarly fails to align, registering a cosine of -0.06 [2]. This detection-intervention gap is not an isolated quirk. The researchers replicated the finding across four models drawn from three distinct families and two parameter scales, ranging from 1B to 9B [2]. The cosine between detection and control directions remained in a tight band of 0.12 to 0.20. Notably, the gap persisted almost unchanged before and after instruction tuning, with cosines of 0.1197 and 0.1200 respectively, suggesting its origin lies in pretraining rather than subsequent fine-tuning [2]. The team explored whether a geometric adjustment could bridge the gap. A 15-degree rotation toward the refusal direction partially succeeded, yielding refusal rates of 73% and 60% on two held-out fake-entity categories while keeping false positives at 1.8% [2]. However, the paper cautions that the cosine itself is not a predictor of steerability. "The cosine is a weight-computable signature of the dissociation between knowing and steering, not a predictor of it," the authors write [2]. Detection, they argue, is a high-dimensional class rather than a single direction, and what separates steerable cases from unsteerable ones is functional, not readable from a static angle [2]. The work lands amid broader efforts to understand and control model behavior. Large language models, which are trained with self-supervised learning on vast text corpora, have grown rapidly in capability and deployment [5]. Families such as Qwen, developed by Alibaba Cloud and distributed under open-source licenses, and DeepSeek, a Chinese firm that launched its R1 model in January 2025, exemplify the global push toward more powerful systems [6][4]. The new geometry findings inject a note of caution into that trajectory: perfect detection does not imply easy control.