SEAL: Searching Expandable Architectures for Incremental Learning

A new framework called SEAL applies neural architecture search to incremental learning, allowing models to expand only when necessary rather than at every new task, according to research posted on arXiv [1][2]. The work, led by Matteo Gambella, targets data-incremental learning, a scenario where disjoint data samples arrive sequentially and are not stored for future access [2]. SEAL adapts the model structure dynamically by expanding it only when a capacity estimation metric indicates it is required [2]. Stability is preserved through cross-distillation training after each expansion step [2]. The neural architecture search component jointly searches for both the architecture and the optimal expansion policy [2]. Existing neural architecture search-based approaches to incremental learning often expand the model at every task, making them impractical in resource-constrained environments [2]. SEAL was designed to address that limitation by allocating additional capacity only when required [2]. Experiments across multiple benchmarks demonstrated that SEAL reduces forgetting and enhances accuracy compared to prior methods [2]. The paper was submitted to arXiv on 15 May 2025 and revised twice, with the latest version posted on 23 June 2026 [1]. The submission sizes for the three versions were 6,576 KB, 6,245 KB, and 6,312 KB, respectively [1]. arXiv, which began on 14 August 1991, is an open-access repository of electronic preprints that are moderated but not peer-reviewed [9]. As of November 2024, the repository receives about 24,000 articles per month [9]. Incremental learning poses a key challenge: balancing plasticity, the ability to learn new tasks, and stability, the ability to preserve past knowledge [2]. Neural architecture search, a branch of automated machine learning, automates the design of deep neural network architectures and has shown success in static settings [2]. The SEAL framework represents an effort to bring that automation to dynamic, sequential learning environments without the resource bloat of earlier expansion-based methods [2].