Closing the Loop on Latent Reasoning via Test-Time Reconstruction

A new test-time training method called ReLAT aims to make opaque latent reasoning in large language models more reliable by checking whether a model’s hidden computations still reflect the original query before generating an answer, according to a paper published on arXiv [1][2]. Recent advances have pushed intermediate reasoning steps out of natural-language text and into latent or cache-level representations, cutting token costs and bypassing a discrete communication bottleneck [1][2]. The trade-off is a loss of inspectability: when reasoning goes latent, there is no straightforward way to verify that the hidden state still honors the constraints of the problem [1][2]. The authors of the paper describe this as an “open loop,” where a latent state is produced and consumed without an input-anchored fidelity check [1][2]. ReLAT, short for Reconstruction-Guided Latent Reasoning At Test Time, attempts to close that loop by using the query itself as a reference [1][2]. The core principle is that if a latent state faithfully represents a query, the query should be recoverable from it; if it cannot be recovered, task-relevant information has been lost [1][2]. The method constructs a differentiable cycle — Question to Latent Thought to Question — and optimizes a query reconstruction loss through the latent thought before answer generation [1][2]. This approach anchors opaque latent computation to the problem specification it is supposed to represent [1][2]. The technique falls under the umbrella of self-supervised learning, a paradigm where a model generates its own supervisory signals from the data rather than relying on external labels [4]. In self-supervised setups, the input is often augmented or transformed to create pairs of related samples, one serving as input and the other formulating the training signal [4]. ReLAT’s reconstruction cycle echoes the logic of an autoencoder, a neural network that learns an encoding function to transform input data and a decoding function to recreate the original input from that encoded representation [3]. Across mathematical reasoning, knowledge QA, and code generation benchmarks on the Qwen model family, ReLAT consistently outperformed single-model inference, text-based collaboration, open-loop latent collaboration, and alternative test-time training objectives [1][2]. On the Qwen3-8B model, ReLAT lifted AIME 2024 accuracy from 56.7% to 73.3%, a 16.6-point gain over the strongest open-loop latent baseline [1][2]. The AIME benchmark is a challenging mathematical reasoning test often used to evaluate advanced problem-solving in large language models, which are trained with self-supervised learning on vast text corpora [9].