Textual Belief States for World Models: Identifiable Representation Learning Under Strict Mediation

A new paper introduces textual latent states and a reinforcement learning method called factorized GRPO to enforce strict mediation in text-based world models, addressing a longstanding identifiability problem in LLM-based architectures [1]. World models operating in partially observed environments depend on latent representations that condense interaction history. In many modern architectures built on large language models, however, predictive performance can be misleading: the decoder can bypass the latent state by directly attending to the full history, making the representation quality untestable [1][2]. The paper, submitted on 26 Jun 2026, formalizes this failure and proposes a solution grounded in a classical principle [1]. Strict latent state mediation requires that all predictions be a function solely of the latent state and the action, not the raw history [2]. Applying this principle to text domains has been an open challenge. Textual latent states are discrete and non-differentiable, which rules out standard variational training, and expressive LLM decoders can easily ignore an information bottleneck [1][2]. The authors introduce textual latent states that are discrete, interpretable, and variable-length. To train them, they develop factorized GRPO, or fGRPO, a tree-structured reinforcement learning method that enforces strict mediation during the learning process [1][2]. The work shows that strict mediation makes representation quality empirically testable, a connection that breaks in history-leaky architectures [2]. Experiments were conducted on the TextWorld and ScienceWorld environments. The results show that one-step prediction accuracy is preserved, while representation quality improves by up to 57% and rollout performance gains reach 98%, with benefits increasing alongside task complexity and horizon [1][2]. The research appears on arXiv, an open-access repository that has hosted e-prints since 1991 and now receives about 24,000 submissions per month [8]. The paper is accessible through the arXivLabs framework, a community innovation space launched in 2020 that allows collaborators to build tools on top of the repository's article pages [7].