Conservation Laws from Data Symmetry in Neural Networks

A theoretical study posted to arXiv on 9 Jun 2026 finds that intrinsic symmetries in training data generally do not create additional conserved quantities during gradient-flow training of neural networks, except in specific cases involving mean squared error loss and data augmentation [1][2]. The paper, titled "Conservation Laws from Data Symmetry in Neural Networks," examines a question at the intersection of dynamical systems and machine learning: whether the symmetries present in a dataset impose corresponding integrals of motion on the network's parameters as they evolve under gradient flow [1][2]. The authors assume the loss function is analytic and non-polynomial, and under that condition they prove that data symmetries generically do not induce any extra conserved quantities [1][2]. An exception emerges when the loss function is mean squared error (MSE). In that setting, the researchers show that data augmentation can yield additional conserved quantities that would not otherwise exist [1][2]. The finding draws a sharp line between the behavior of networks trained with MSE and those trained under more general loss functions. To formalize the phenomenon, the authors introduce the concept of tensorizable networks. These are architectures whose dependence on parameters and inputs can be separated using an intermediate representation [1][2]. The family includes linear and polynomial networks, as well as Lightning Attention, a recent attention mechanism [1][2]. By working within this framework, the study provides a structured way to analyze when and why conservation laws appear. The work sits within a broader mathematical tradition of studying conserved quantities in dynamical systems. Partial differential equations, which describe everything from fluid dynamics to quantum mechanics, are often analyzed through their invariants and symmetries, though no universal theory exists for all PDE types [3]. The search for invariants in neural network training dynamics mirrors that tradition, applying tools from classical mechanics and differential geometry to modern optimization. The paper appears as a preprint on arXiv, a platform that has become a primary distribution channel for machine-learning research. arXivLabs, a framework for experimental community projects, supports the development of new features on the site, though the study itself is a conventional research article [1]. The authors have not yet announced peer-reviewed publication.