How Deep Are Deep GPs, Really? A Sharp Threshold and a Non-Gaussian Limit for Compositional GPs
Researchers have identified a critical threshold in compositional Gaussian Processes (GPs) that determines whether their behavior in deep models becomes degenerate or non-trivial. The study establishes a sharp bandwidth threshold, $r_c(d) = \Theta(\sqrt{d})$, above which the GP prior converges to constant functions. Below this threshold, the prior converges to non-Gaussian, non-degenerate distributions, offering a more useful probabilistic model for deep Bayesian networks. AI
IMPACT Identifies a critical threshold for deep Gaussian Processes, potentially enabling more robust Bayesian modeling in deep learning architectures.