OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems

A new regularization technique called OrthoReg aims to prevent hybrid symbolic-neural models from redundantly relearning known physics, according to a paper submitted in 2026 [1]. The method directly penalizes overlap between a system’s symbolic and neural components, a problem that standard L^2 regularization fails to address when the symbolic structure is itself discovered from data [1]. Hybrid modeling seeks to combine the interpretability of prescribed, physics-based equations with the flexibility of neural networks [1]. A persistent difficulty is that the neural network can inadvertently absorb the mechanistic relationships the symbolic part is meant to capture, producing models that are both redundant and difficult to interpret [1]. This issue is especially acute when the symbolic component is not manually specified but learned through sparse discovery, a process that identifies governing equations directly from data [1]. Existing approaches often rely on L^2 regularization, which penalizes the magnitude of the neural network’s weights. The authors of the new work argue that the theoretical justification for this strategy depends on a projection argument that collapses when the symbolic model is discovered rather than prescribed, allowing the neural residual to overlap with the recovered structure [1]. OrthoReg, short for Orthogonal Regularization, is designed to enforce a complementary decomposition. The symbolic part is constrained to express only what the candidate function library can represent, while the neural part is forced to model the remaining dynamics [1]. In tests on benchmark dynamical systems with partial library mismatch — scenarios where the true governing equations are not fully contained in the candidate library — OrthoReg improved both the recovery of the correct symbolic terms and the model’s performance on out-of-distribution data [1]. Dynamical systems modeling underpins a wide range of scientific and engineering disciplines, from climate simulation to biological pathway analysis. The challenge of balancing model fidelity with physical consistency has driven interest in hybrid architectures across fields. For instance, in catalysis research, transfer learning and joint training across datasets have been explored to improve model generalization when consolidating data from varied computational methods [4]. The broader machine-learning community continues to develop tools for discovering and verifying such hybrid models, with platforms like arXivLabs supporting experimental features for code discovery and citation analysis that accompany new preprints [1].