Knowledge Graph Enhanced Memory-Augmented Retrieval for Long Context Modeling

A new framework called KGERMAR constructs dynamic knowledge graphs from input text during inference to improve how language models handle long documents, according to research submitted to arXiv on June 12, 2026 [1]. The approach addresses a core weakness in existing memory-augmented retrieval systems: their reliance on semantic similarity alone, which can miss explicit structural relationships between entities across thousands of tokens [1]. The framework, formally titled Knowledge Graph Enhanced Memory-Augmented Retrieval for Long Context Modeling, performs real-time entity and relation extraction to build context-specific knowledge graphs as it processes text [1]. This contrasts with systems that depend on static knowledge bases, allowing the model to adapt its retrieval to the specific document collection at hand [2]. The key insight, the authors write, is that "the same input text used for language modeling contains rich structural information—entities, their types, and the relationships between them—that can be extracted in real-time to build a domain-adaptive knowledge graph" [3]. KGERMAR maintains three specialized memory banks—contextual, semantic, and structural—each updated incrementally as new contexts arrive and managed via a least-recently-used eviction policy when capacity is reached [9]. Retrieval signals from all three banks are combined using learned fusion weights, and the resulting representations are injected into the language model through retrieval causal attention at designated upper layers [9]. This architecture distinguishes KGERMAR from purely semantic predecessors such as ERMAR and MemLong, enabling the system to differentiate between passages that mention related terms coincidentally and those that describe explicit relationships between the same entities [3]. The framework was evaluated on four datasets: SlimPajama with 84,700 training examples, WikiText-103 with 4,358 examples, PG-19 with 100 examples, and Proof-pile with 46,300 examples [1]. Across context lengths ranging from 1,000 to 32,000 tokens, KGERMAR achieved up to 8.5 percent lower perplexity and 2 to 2.5 times better memory efficiency compared to memory-augmented baselines [1]. The researchers also report superior in-context learning performance across five natural language understanding tasks [2]. Memory-augmented retrieval sits within the broader category of retrieval-augmented generation, a technique first proposed in 2020 that enables language models to pull relevant information from external data sources before generating responses [6]. Such methods have been widely adopted to reduce factual errors and allow models to cite sources, providing greater transparency for users [6]. KGERMAR extends this lineage by adding a structural dimension to the retrieval process, addressing what the authors describe as a critical gap in long-context settings where the relevant passage may appear thousands of tokens earlier and share only structural, not lexical, similarity with the query [4]. The paper was submitted to the ACL ARR 2026 January review cycle under the research area of Retrieval-Augmented Language Models [5]. The submission is classified as an NLP engineering experiment focusing on English-language data [5].