Forget to Know, Remember to Use: Context-Aware Unlearning for Large Language Models

A team of researchers has identified a flaw in how large language models forget information, finding that current unlearning methods strip a model’s ability to use deleted knowledge even when it is later provided as context, and propose a new technique to restore that capability [1]. The study, led by Yuefeng Peng and submitted to the arXiv preprint server, addresses a growing challenge in artificial intelligence: removing sensitive or outdated data from large language models without a costly full retraining [1]. The process, known as unlearning, is designed to excise specific knowledge while preserving the model's overall performance on other tasks [2]. The first version of the paper was submitted on 20 Oct 2025, with a revised version following on 27 May 2026 [1]. However, the researchers found that standard evaluations of unlearning miss a critical usability dimension. While existing methods are tested on how completely they forget targeted information and how well they retain other knowledge, they are not tested on what the authors call "contextual utility" [2]. This refers to a scenario where a user might intentionally provide the previously deleted information within a new prompt, expecting the model to reason with it. A systematic evaluation of six state-of-the-art unlearning methods revealed they consistently impair this ability [2]. The problem loosely parallels a well-documented phenomenon in biological memory known as spontaneous recovery, where a previously extinguished conditioned response can re-emerge after a delay [3]. In the context of machine learning, the goal is not to prevent re-emergence but to ensure the model can fluidly switch between its internal, unlearned state and processing the same information when it is presented externally. This requires a form of cued recall, a memory retrieval process triggered by a specific stimulus, rather than a total erasure of the underlying concept [4]. To solve this, the team augmented existing unlearning objectives with a plug-in component designed to preserve the model's ability to use forgotten knowledge when it appears in context. Their experiments showed the approach restores contextual utility to near original levels while still maintaining effective forgetting and utility on the retain set [2]. The work was developed within the arXivLabs framework, a platform for community collaborators to build new features on the arXiv website [1].