Edge of Stability Selectively Shapes Learning Across the Data Distribution

The edge of stability, a regime long treated as a uniform constraint on neural network training, selectively redistributes learning across data subgroups, amplifying progress for some while suppressing it for others, according to new research submitted to arXiv on 2 June 2026 [1]. The paper challenges the prevailing view that the edge of stability (EoS) acts as a global property of optimization. Instead, the authors demonstrate that the stability constraint functions as a mechanism governing the allocation of learning across the training distribution [1]. Using a branching intervention that enters or exits the EoS regime from an identical training state, the researchers causally established a trade-off: subsets of data that benefit under EoS do so at the expense of other subsets [2]. The study identifies two necessary conditions for a data subgroup to gain an advantage. First, the group’s aggregate gradient must align with the top Hessian eigenvector. The team isolated this mechanism through a controlled perturbation that preserved distance but randomized direction, which destroyed the alignment and eliminated the advantage [3]. Second, the group must sustain a non-vanishing gradient magnitude over time. Under cross-entropy loss, gradient saturation decouples confidently classified groups, shifting the advantage to output-outliers whose gradients persist [4]. These findings connect to a broader debate on whether low curvature improves generalization. The EoS regime, in which sharpness is actively constrained near the stability threshold, provides a natural setting to test what low curvature actually confers. The results suggest the answer is not global: the functional benefit depends on which subset dominates the top Hessian eigendirection and shifts with the geometric composition of the training distribution. Flatness, in this view, is a directional property determined by data geometry rather than a scalar property of the solution [3]. A separate theoretical study on two-layer ReLU networks trained below the edge of stability offers a complementary perspective. That work introduces the concept of data shatterability—how easily a data distribution can be partitioned into many disjoint small regions by ReLU half-spaces—as a key geometric quantity controlling implicit regularization. For data supported on a mixture of low-dimensional balls, generalization bounds adapt to the intrinsic dimension rather than the ambient dimension. For isotropic distributions, rates deteriorate as probability mass concentrates toward the unit sphere, where gradient descent favors memorization over learning shared patterns [5]. The new EoS findings replicate qualitatively across different architectures and optimizers, according to the paper’s appendix [4]. The authors frame the work as a step toward connecting two largely separate lines of inquiry: implicit regularization in parameter space and inductive bias over the data distribution [3].