Memory-Managed Long-Context Attention: A Preliminary Study of Editable Request-Local Memory

A new study proposes separating a language model’s fast sequence processor from an explicit, editable memory to fix a long-standing conflation in long-context attention, according to a preprint posted to arXiv on 27 June 2026 [1]. The work argues that linear, recurrent, and sparse attention mechanisms efficiently compress history but lack rules for when a fact should be written, overwritten, protected from distractors, or discarded [1]. The authors introduce “memory-managed long-context attention,” which pairs a fast recurrent or sparse backbone with request-local memory slots and a query-time sparse fallback [2]. Across structured synthetic tasks, token/chunk/sequence bridges, generated natural language, and frozen-model diagnostics, pure fixed-state or pure sparse methods failed on overwrite, version, anti-pollution, or no-write-signal cases, while a hybrid design covered both routes [2]. A small mechanism stress test spanning 2,097,152 tokens reached 50/50 pooled accuracy with between 2 and 132 active chunks [3]. A 2.74M-parameter minimal causal event-token model achieved 595/600 with lite write supervision, which the authors present as proof of trainability rather than evidence of scale [3]. A six-family frozen-hidden-state bridge reached 1079/1080 controlled pointer accuracy, but the setup relied on generator-provided integer key IDs and separately encoded canonical key strings, making it an oracle-metadata probe rather than an open-text entity resolver [2]. Local RULER 4K diagnostics remained close to full context, while a 33-record LongBench v1 16K subset showed that naive lexical selection does not generalize [3]. The paper separates three claims: controlled slot lifecycle is feasible, sparse fallback is necessary when writes lack future-query signals, and learned open-domain selection remains the main architectural bottleneck [1]. The authors do not claim a final generative architecture, global slot-trajectory convergence, or systems superiority [2]. Related work has explored externalizing language-model memory in different ways. MemLong stores past contexts in a non-trainable memory bank and retrieves chunk-level key-value pairs, keeping lower layers frozen during training to avoid distribution shift [5]. Attention-state memory constructs a per-layer dictionary of precomputed attention outputs indexed by clustered query vectors, removing the need to attend to a prefix during inference and reducing attention latency by 1.36× at 8K on in-context learning benchmarks [4]. The new study differs by making the memory slots explicitly editable and coupling them with a lifecycle policy, rather than treating memory as a static lookup table [1].