Representational Depth of Evaluation Awareness Shifts With Scale in Open-Weight Language Models

A new study finds that the layer inside open-weight language models where researchers can detect “evaluation-awareness” shifts systematically as models grow larger, moving from late layers in small models to early layers in large ones, complicating the interpretation of safety benchmarks. The paper, posted to arXiv on 28 June 2026, examines whether language models recognize when they are being tested — a question with direct implications for AI safety, because a model that detects an evaluation context could alter its behavior and make downstream benchmarks harder to trust [1][2]. The authors probed 11 models across three families: Qwen 2.5, Gemma 2, and Llama 3.2 [1][2]. In both the Qwen 2.5 and Gemma 2 families, the layer at which evaluation-awareness was most linearly recoverable migrated from later layers in smaller variants to earlier layers in larger ones [1][2]. “This suggests that scale changes not only the strength of evaluation-awareness but also where it is most linearly recoverable in the network,” the researchers write [2]. The finding helps explain why within-family scaling trajectories were non-monotonic or inverse rather than smooth and family-general, undercutting the assumption that a simple universal power-law account holds under denser within-family sampling [1][2]. Large language models — systems trained with self-supervised learning on vast text corpora — have become central to natural language processing tasks such as generation, reasoning, and parsing [10]. The three model families examined in the study are widely used in both research and deployment. Llama 3.2, for instance, has shown competitive performance in semantic tasks including AMR parsing, where straightforward fine-tuning achieved a SMATCH F1 of 0.804 on the LDC2020T02 test set [7]. Qwen 2.5 and Gemma 2 have similarly been evaluated across reasoning benchmarks such as GSM8K, ARC, and SVAMP, where internal entropy dynamics were found to correlate with answer correctness [6]. The new work distinguishes between white-box probe signals — derived from direct access to internal representations — and black-box behavioral expression. Across all families, probe signals were consistently stronger than what the models expressed in their outward behavior, and the relationship between the two varied by family in ways not predicted by probe AUROC alone [1][2]. AI safety concerns have grown alongside the capabilities of generative models. Since the 2020s, advances in generative AI have enabled the creation and modification of media, while prompting discussions about regulation, unintended consequences, and potential existential risks [3]. The study’s authors argue that understanding how and where evaluation-awareness is represented inside models is a prerequisite for building benchmarks that remain informative as systems scale [1][2].