Multi-Turn Reasoning When Context Arrives in Pieces: Scalable Sharding and Memory-Augmented RL

Researchers have identified a sharp decline in large language model accuracy when task-critical information is spread across multiple conversation turns, a phenomenon they call “Lost in Conversation.” A new preprint proposes training models with a compact rolling memory to reverse the degradation, which can reach 65 percent. [1] The work, posted to the arXiv preprint server on June 11, 2026, demonstrates that standard large language models (LLMs)—neural networks trained on vast text corpora for generation and analysis [8]—struggle when a user parcels out essential details over several exchanges. Accuracy falls by up to 65 percent even though the full conversation history remains available to the model. [1] The authors attribute the failure to the way models attend to a continuously expanding dialogue log. [2] To address the problem, the team trained models to maintain a compact rolling memory that compresses past turns rather than rereading the entire history. [1] Making such training practical required a low-cost data-generation method. The researchers built a sharding pipeline that automatically converts single-turn question-answering datasets into multi-turn episodes where information arrives in fragments, removing the need for hours of manual annotation. [2] Training was conducted solely on a sharded version of GSM8K, a grade-school math benchmark. [1] The resulting memory-augmented policy not only improved multi-turn accuracy on the training domain but also generalized zero-shot to harder math problems and out-of-domain long-context question-answering tasks. [2] In a striking result, the memory-trained models outperformed baselines that had access to the full conversation history at test time, indicating that learning to compress information fosters more robust incremental reasoning than exposure to the complete context alone. [1] The preprint appears on arXiv, an open-access repository that hosts e-prints across physics, mathematics, computer science, and related fields. [6] As of late 2024, the platform was receiving roughly 24,000 new submissions per month. [6] The paper’s landing page also surfaces experimental community tools through the arXivLabs framework, which allows third-party developers to build features such as citation explorers and recommender systems directly on article pages. [4][5]