Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents

A new study identifies a “memory-update gap” in large language model agents, showing they systematically fail to discard outdated facts and use current ones during long, multi-session interactions, a flaw that persists even in frontier models. The research, posted to the arXiv preprint server on June 25, isolates the failure on real conversational data and demonstrates it is a distinct, unsolved problem [1]. When a frontier model, gpt-5.4, was given full context of a conversation, it answered questions about updated facts with 92% accuracy. Replacing that full context with a bounded, self-maintained memory caused accuracy to drop to 77%, a statistically significant gap with a paired McNemar p-value below 0.005 [1]. The authors note that the bottleneck is memory maintenance, not comprehension, and that using a stronger model does not close the gap [1]. The failure is not caused by an undersized memory. As the conversation length grew by a factor of 24, accuracy fell from 68% to 28%. Giving the agent proportionally more memory yielded no detectable recovery, with accuracy remaining at 28% across 25 trials [1]. The paper concludes that the failure scales with the length of the conversation, not the compression ratio [1]. The team released Supersede, an open reinforcement-learning environment built on the verifiers and prime-rl stack, which turns the measurement into a training signal. Agents are rewarded for answering from the current value of a fact and penalized for using stale ones [1]. Using GRPO fine-tuning on a small open model, Qwen2.5-3B, the researchers nearly doubled its held-out supersession accuracy on real, unseen conversations, from 9.0% to 16.7% in a single run [1]. A monotonic checkpoint curve indicated that the learned policy, not the testing harness, carried the gain [1]. The paper appears on arXiv, an open-access repository that hosts preprints across physics, computer science, and other fields and has grown to receive roughly 24,000 submissions per month as of late 2024 [7]. The study’s findings add to a growing body of work examining the reliability of LLM agents in dynamic environments where factual information changes over time.