Uncertainty Estimation and Generalization Bounds for Modern Deep Learning
This thesis explores the application of Bayesian principles to enhance the understanding of modern deep learning systems, focusing on generalization and uncertainty quantification. It introduces the Deep Variational Implicit Process (DVIP) as a scalable Bayesian framework for deep architectures and proposes post-hoc methods like Variational Linearized Laplace Approximation (VaLLA) and Fixed-Mean Gaussian Process (FMGP) to add calibrated uncertainty estimates to pre-trained networks. Theoretically, it connects diversity, smoothness, and stochasticity within a probabilistic framework using PAC-Bayesian and large-deviation theory to explain the generalization capabilities of over-parameterized neural networks. AI
IMPACT Introduces novel methods for uncertainty estimation and theoretical frameworks for understanding deep learning generalization.