The strength of clinical evidence is recoverable from language model representations but not from their stated grades

Large language models encode the strength of clinical evidence in their internal representations but fail to state that grade when asked, according to a study that tested 22 open-weight models across 20,611 harmonized clinical claims [1]. The work, led by Soroosh Tayebi Arasteh and posted to arXiv on 27 June 2026, compiled 45,134 clinical claims from six public sources and harmonized 20,611 of them into a four-level evidence grade using three independent frameworks [1]. Researchers then probed 22 local, open-weight LLMs ranging from 0.6 to 70 billion parameters, spanning general, medical, and reasoning-tuned architectures [1]. A linear estimator recovered the evidence grade from model activations in every model tested, with a median AUROC of 71.8 [1]. The same models, however, could not verbalize that grade: their stated assessments fell to chance, landing 25 to 27 percentage points below the estimator’s performance [1]. The recoverable signal was distinct from factual truth and still flagged weakly supported claims, achieving an AUROC of 69.2 on that subset [1]. But the signal was largely lexical — it reflected the vocabulary in which a claim was phrased rather than a deeper encoding of evidentiary support, and it did not transfer across clinical topics or grading frameworks [3][4]. “The evidence-grade signal these models carry is thus real and robustly recoverable, but largely lexical,” the authors write, noting that almost none of the signal survives once vocabulary is accounted for [4]. Contrary to expectations that larger or more specialized models would hold cleaner internal structure, decodability did not rise with scale and was weakest in reasoning-tuned models [3]. This dissociation between what a model internally registers and what it states aligns with broader findings in medical AI. A separate benchmark, MedSynth, found that even the best-performing model achieved an average score of only 44.59 on evidence synthesis tasks, with models struggling to assess the certainty of supporting evidence and to assign appropriate recommendation strength [9]. Another evaluation of LLMs on systematic review conclusions reported that models are “generally reluctant to express uncertainty, often committing to a more certain outcome that appears plausible” [10]. Those behavioral patterns reinforce the central finding of the arXiv study: clinical LLMs carry an ordered evidence-strength signal they do not express, leaving their stated grades unable to convey how strongly a claim is supported even when that information is recoverable from their representations and text [1][4].