One-Step Generalization Ratio Guided Optimization for Domain Generalization

A new optimizer called GENIE aims to improve how machine learning models handle unseen data by preventing a small subset of parameters from dominating the training process, according to a paper submitted in 2026 [1]. The technique targets a persistent challenge known as Domain Generalization (DG), where models trained on specific source domains often fail when applied to new, unseen target domains because they latch onto spurious correlations rather than learning broadly applicable features [1]. Gradient-based DG methods, which guide model updates in a dominant direction, can inadvertently reinforce these misleading patterns [2]. While recent approaches have used dropout to regularize overconfident parameters, they have not explicitly adjusted gradient alignment or ensured balanced updates across all parameters [3]. The new optimizer, named GENIE—short for Generalization-ENhancing Iterative Equalizer—introduces a metric called the One-Step Generalization Ratio (OSGR) to measure how effectively a single gradient update reduces test loss compared to training loss, providing insight into each parameter’s contribution to generalization [4]. By dynamically equalizing OSGR through a preconditioning factor, GENIE prevents a small subset of parameters from dominating optimization, which the authors argue promotes the learning of domain-invariant features [5]. The theoretical framework shows that existing optimizers typically focus on either convergence speed or gradient alignment, often resulting in suboptimal generalization, whereas GENIE explicitly balances both objectives [4]. This balancing act is conceptually related to multi-objective optimization, a field concerned with finding solutions when trade-offs exist between conflicting goals [6]. The authors report that GENIE retains the convergence rate of standard Stochastic Gradient Descent (SGD) in non-convex settings while achieving a higher OSGR [4]. Empirical validation was conducted on five standard DG datasets, where GENIE consistently outperformed established optimizers, even with extended iterations, and it enhanced performance when integrated with existing DG and single-DG algorithms [4]. The work was presented as an oral at the 2025 International Conference on Machine Learning (ICML) [5].