Deployment-Time Memorization in Foundation-Model Agents

Researchers have mapped a new privacy frontier for artificial intelligence agents, finding that the persistent memories these systems build about users must be treated as a first-class memorization mechanism, distinct from the data stored in their underlying models. The study, posted to arXiv on June 8, 2026, examines what its authors call "deployment-time memorization" in foundation-model agents — long-lived systems designed to remember user details across interactions [1]. Unlike traditional large language models, which are neural networks trained on vast text corpora to generate and summarize language, these agents accumulate personal data as an explicit function of their operation [1][3]. The work formulates agent memory as a privacy-utility frontier, measured by two metrics: Personalization Recall, which tracks how well an agent tailors its behavior, and Adversarial Extraction Rate, which quantifies how much sensitive information an attacker can pull from the system [1][2]. A third metric, the Forgetting Residue Score, was introduced to determine whether deleted information remains recoverable from derived memory tiers [2]. The researchers swept three design variables — summarization aggressiveness, retrieval breadth, and deletion mode — using the LongMemEval benchmark [1]. They found that key-fact summarization reduced canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini, while preserving nearly all personalization recall [1][2]. Once content was compressed, widening the retrieval window did not restore the leakage [1]. However, the same compression created a deletion-fidelity failure. When only raw memory was purged, derived summary copies remained recoverable in approximately 20% of instances [1][2]. Only a full-pipeline purge or tombstone redaction drove the worst-tier residue to zero [1][2]. The findings arrive as AI agents become more embedded in daily life. ChatGPT, the generative chatbot developed by OpenAI, reached 100 million monthly active users within two months of its 2022 launch and counted 900 million weekly active users by February 2026 [5]. The broader AI boom has also seen new entrants such as DeepSeek, a Chinese firm that released its R1 model in January 2025 with training costs it claimed were a fraction of those for comparable Western models [8]. The paper argues that persistent agent memory should be assessed by what it helps agents recall, what it makes extractable, and what it can truly erase [1][2].