The conditional-mean barrier: From deterministic regression to conditional distribution learning

A newly published tutorial defines a hard boundary for squared-loss predictors, termed the conditional-mean barrier, beyond which remaining prediction error is irreducible aleatoric variance and cannot be reduced without switching to distributional loss functions [1]. The tutorial, posted to arXiv, addresses a common failure mode in computational science and engineering where deterministic machine-learning surrogates learn a well-defined mathematical object yet miss application-relevant uncertainty. This occurs in one-to-many settings such as coarse graining, partial observation, or inverse reconstruction, where a single input may correspond to multiple valid outputs [1]. The authors frame the problem around the conditional-mean barrier: the point at which a squared-loss predictor has reached the conditional mean and the remaining error is irreducible aleatoric variance [1]. Two diagnostics are provided for locating this barrier: residual-feature orthogonality and the coefficient of determination measured against its explained-variance ceiling [1]. The authors prove that adding latent randomness to a squared-loss predictor collapses it back to the conditional mean, meaning that simply injecting noise into a deterministic model does not cross the barrier [1]. Crossing it requires a loss function that scores entire distributions rather than point predictions [1]. The concept connects to formal definitions of randomness and predictability. In probability theory, randomness is not haphazardness but a measure of uncertainty of an outcome, and individual random events are unpredictable even when their frequency distribution is known [4]. Predictability itself is defined as the degree to which a correct forecast of a system's state can be made [3]. The tutorial's barrier marks the precise limit of that predictability under squared loss. The paper organizes common distributional objectives by the feature of the conditional law each targets. These include negative log-likelihood, moment and observable matching, variational objectives, adversarial divergences, and score matching [1]. The emphasis remains on the boundary itself and a finite-data procedure for recognizing it, rather than a survey of methods beyond it [1]. CPU-based demonstrations on a two-branch law and a two-scale Lorenz-96 closure problem show how the diagnostics distinguish deterministic underfitting from residual distributional variability [1]. The Lorenz-96 model is a standard testbed for chaos and predictability studies, making it a natural choice for illustrating where a deterministic surrogate plateaus and distributional methods become necessary [1].