EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments

A new benchmark suite called EvoArena reveals that large language model agents struggle in dynamic environments, achieving an average accuracy of just 39.6%. Researchers also propose EvoMem, a patch-based memory system that modestly improves performance by helping agents track environmental changes over time. Most evaluations of large language model agents assume static conditions, but real-world deployment requires continual adaptation to shifting environments and task conditions [1]. To address this gap, the researchers behind EvoArena modeled environment changes as sequences of progressive updates across terminal, software, and social domains [1][2]. The benchmark tests whether agents can align their knowledge, skills, and behavior as conditions evolve [2]. Current agents performed poorly on the suite, reaching an average accuracy of 39.6% across the three evolving domains [1][2]. The team introduced EvoMem, a memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental change through changes in their own memory [1][2]. EvoMem yielded an average gain of 1.5% on EvoArena [1][2]. On established static benchmarks, EvoMem also provided benefits. Performance on the GAIA benchmark improved by 6.1%, while scores on LoCoMo rose by 4.8% [1][2]. The system further improved chain-level accuracy by 3.7% on EvoArena, a metric that measures success across consecutive sequences of related evolutionary subtasks [1][2]. Mechanistic analysis indicated that EvoMem improves evidence capture in memory, suggesting better preservation of complete evolving environment states [1][2]. The findings underscore the importance of modeling evolution in both evaluation and memory design for reliable agent deployment [1][2]. The work was submitted to arXiv on June 11, 2026 [1].