When Role-playing, Do Models Believe What They Say?

Researchers have made new findings on the internal workings of language models, particularly how they handle role-playing and internalized beliefs.

Recent studies have investigated the behavior of language models when role-playing characters with beliefs different from modern consensus. One study, submitted to arXiv on June 9, 2026^[1], found that language models can state contradictory beliefs when role-playing. The study used various methods to induce personas, including prompting, in-context learning, and supervised fine-tuning. The researchers measured belief internalization using truth probes and behavioral tests, finding a spectrum of internalization across different approaches. Meanwhile, another study submitted on June 26, 2026, introduced a new metric, SCSuff, to evaluate the sufficiency of free-text explanations in large language models (LLMs)^[2]. LLMs are increasingly used in high-stakes domains, and understanding how they justify their outputs is crucial. The SCSuff metric agrees with targeted perturbation tests where applicable. A third study, also submitted on June 26, 2026, explored the ability of models to predict downstream consequences based on edited intermediate states^[3]. The researchers found that some models can make such predictions, depending on both the edited state and the current move.