Why Limit the Residual Stream to Layers and Not Tokens? Persistent Memory for Continuous Latent Reasoning

A new model architecture called AGCLR addresses a critical weakness in continuous latent reasoning systems, where models lose earlier factual information as they process deeper reasoning chains, according to research published June 5 on arXiv [1]. Large language models have shown strong performance on mathematical and multi-hop planning tasks [1]. The CoCoNuT paradigm, short for Chain of Continuous Thought, extended these capabilities by allowing models to reason in latent space and explore multiple reasoning paths simultaneously rather than committing to a single chain early in the process [1]. However, researchers identified a limitation they call the concept bottleneck. At each reasoning pass, intermediate hidden states are overwritten, causing the model to lose critical facts computed in earlier steps as reasoning depth increases [1]. The effect is measurable. On the HotpotQA dataset, vanilla CoCoNuT achieved a 10.4% exact match score, failing to improve over the chain-of-thought baseline of 11.0% [1]. Performance also degraded with curriculum depth on the GSM8K dataset [1]. To address this, the researchers propose AGCLR, which stands for Adaptive Gated Continuous Latent Reasoning [1]. The architecture augments CoCoNuT with a Gated Concept Stream, a persistent residual memory maintained across all reasoning passes [1]. Three learned gates control the memory: a write gate that commits intermediate facts to memory, a read gate that retrieves relevant prior states, and a forget gate that prunes irrelevant context [1]. The design draws a parallel to how biological systems manage sequential information processing. In molecular biology, transcription factors regulate gene expression by binding to specific DNA sequences, ensuring genes are activated or repressed at the correct time and in the correct amount throughout a cell's life [7]. The gating mechanism in AGCLR performs an analogous function for artificial reasoning, selectively retaining or discarding information as computation proceeds. Evaluated on GSM8K, HotpotQA, and ProsQA using GPT-2 as the base model, AGCLR achieved consistent improvements across all dataset types [1]. The performance gap between AGCLR and the baseline compounds as curriculum depth increases, indicating the architecture directly resolves the concept bottleneck rather than merely masking its symptoms [1]. Code for the model is available through an anonymous repository [1].