Continual Learning in Modern Hopfield Networks with an Application to Diffusion Models

A new study investigates how generative AI models forget previously learned information when trained on new tasks, using a framework derived from modern Hopfield networks to predict which data points are most vulnerable to loss. The research, submitted on 27 May 2026, addresses a gap in understanding continual learning within generative models such as diffusion models, which are increasingly deployed as foundation models and adapted through sequential fine-tuning [1][2]. The authors introduce a concept called "intrinsic forgetting," defined as an increase in Hopfield energy after a task change [2]. By leveraging recent theoretical links between modern Hopfield networks (MHNs) and diffusion models, the team transferred analyses from the more tractable MHN setting to complex generative systems [1][2]. In controlled MHN experiments, the researchers proved that high-energy, outlier-like samples experience a larger energy increase than cluster-like samples, indicating that data points in sharp, isolated basins are more forgettable [2]. This finding suggests that the geometry of the learned distribution directly influences what a model loses first. The study further examined memory replay, a common technique to mitigate forgetting, and demonstrated that replay is particularly effective for these high-energy samples, enabling an energy-based strategy for selecting which past data to rehearse [2]. The predictions were validated on two diffusion models under continual-learning conditions: Stable Diffusion and a pixel-space DDPM [1][2]. In both models, Hopfield energy tracked reconstruction-based forgetting, and replay experiments confirmed energy-dependent mitigation consistent with the MHN analysis [2]. Diffusion models, first described in 2015, have since become the basis for image generation systems like DALL-E, while the broader field of neural networks has evolved from early perceptrons through an "AI winter" to the current proliferation of deep learning architectures [5][4]. Modern neural networks rely on layers of artificial neurons that process signals through weighted connections, with training accelerated by GPUs and large datasets [4]. The study's approach offers a principled method for prioritizing replay samples, moving beyond random selection toward energy-guided retention of outlier-like information.