How Deep Are Deep GPs, Really? A Sharp Threshold and a Non-Gaussian Limit for Compositional GPs

New research on deep Gaussian processes identifies a sharp bandwidth threshold that separates degenerate behavior from a non-trivial, non-Gaussian limit as the number of layers grows, challenging prior assumptions about the utility of deep compositional priors. The study, posted to arXiv on June 6, examines compositional priors — the generic properties of layered functions in deep Bayesian models where each layer is a vector-valued Gaussian process [1][2]. Earlier work had shown that for the radial basis function kernel, the prior degenerates as depth increases across a certain range of bandwidths, collapsing to a set of constant functions that offer little value as a probabilistic model [2]. The new paper sharpens that picture considerably. The authors establish a bandwidth threshold r_c(d) = Θ(√d), where d is the dimensionality of the input space [1][2]. Above this threshold, the limit remains degenerate, confirming and tightening prior bounds. Below it, however, the prior converges to a limit distribution π_Z̄ that is both non-degenerate and non-Gaussian, with non-vanishing dependence between coordinates [1][2]. “In contrast to the previously known degenerate regime, deep Gaussian process priors can therefore admit non-trivial limits,” the authors write [2]. The empirical section verifies the threshold across a range of dimensions and reveals complex multimodal behavior in the limit distributions — a regime that narrows as d increases and would be difficult to locate without the theoretical threshold [1][2]. The findings carry implications for the design of deep Bayesian models, which are a canonical example of neural networks with random weights [1]. While the paper focuses on theoretical properties of priors, the broader landscape of AI model evaluation has drawn scrutiny from regulators and researchers. A separate analysis of documentation from five frontier models — including Claude 4.5, developed by Anthropic, and GPT-5 — found that safety-critical disclosures such as deception behaviors, hallucinations, and child safety evaluations accounted for 148, 124, and 116 aggregate points lost, respectively, across all evaluated models [3]. Anthropic, founded in 2021 by former OpenAI members and valued at an estimated $965 billion as of May 2026, trains its Claude series using “constitutional AI” to improve ethical and legal compliance [5][6]. Microsoft, another major player in the AI space, has expanded its cloud computing and artificial intelligence offerings under CEO Satya Nadella, who took the role in 2014 [7]. The arXiv study does not address these commercial systems directly, but its theoretical results on the limiting behavior of deep Gaussian processes may inform future work on model architecture and uncertainty quantification [1][2].