The limits of interpretability in multiple linear regression

Multiple linear regression, long viewed as a transparent alternative to opaque neural networks, can produce misleading physical interpretations when input features are correlated, according to a theoretical analysis submitted to arXiv on June 14, 2026 [1]. The study examines how multicollinearity — strong correlations among input features — undermines the interpretability of linear models [1]. Researchers found that learned weights can exhibit large dataset-to-dataset fluctuations and oscillatory behavior across physically similar features, making their interpretation difficult or even impossible [1]. Although the instability of weights under multicollinearity is well known in statistics, its consequences for physical interpretation, particularly the connection to oscillatory weights, had not been systematically clarified [2]. The authors analyzed the eigenmodes of the feature correlation matrix and demonstrated that small-eigenvalue modes amplify fluctuations in the weights and generate oscillatory patterns that do not necessarily reflect meaningful contributions [3]. Linear regression models relationships using linear predictor functions whose unknown parameters are estimated from data, and they are often fitted using the least squares approach [6]. When features are collinear, however, the classic interpretation of the coefficient value is not realistic and cannot be considered to explain the model [5]. Ridge regularization, a technique that adds a penalty to the least squares cost function, can suppress these unstable modes [4]. But the paper cautions that Ridge does not provide a fundamental justification for interpreting the fitted weights as physical feature importances. The resulting weight pattern can change drastically with the regularization strength, even when prediction performance remains essentially unchanged [3]. This suggests that the conventional criterion for choosing the regularization parameter, based on maximizing test-performance scores, does not necessarily lead to an optimal strength from the viewpoint of interpretation [4]. The findings extend beyond physics. The researchers confirmed the generality of their results by analyzing a diverse collection of publicly available datasets [1]. A separate 2024 paper argued that linear regression models should be treated equally to complex models when it comes to explainability and interpretability, noting that multicollinearity is one of the main issues to explain any machine learning model [5]. Polynomial regression, which models relationships as polynomials, is itself a special case of multiple linear regression and faces the same challenges [8]. The new work clarifies why, in the presence of multicollinearity, physical interpretation can remain difficult even for models long assumed to be inherently interpretable [1].