Bayesian neural networks: Likelihood tempering impact analyzed

Researchers have investigated the impact of likelihood tempering on variational Bayesian linear neural networks. They found that in wide networks, Gaussian mean-field variational inference can lead to "prior dominance," where the variational predictive distribution collapses to the prior predictive as network width increases. The study derives the limiting predictive distribution for single-hidden-layer linear networks with isotropic Gaussian priors under specific tempering schedules, revealing phase transitions for predictive expectation and variance at different scales. The findings suggest that with appropriate choices of tempering parameters, it's possible to recover either the neural network Gaussian process posterior expectation or its variance. AI

IMPACT Provides theoretical insights into the behavior of Bayesian neural networks, potentially informing future model architectures and training techniques.

RANK_REASON This is a research paper detailing theoretical findings on Bayesian neural networks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Bayesian Neural Networks
• evidence lower bound
• Gaussian function
• Gaussian process
• Kullback--Leibler divergence
• least squares method
• Neural network Gaussian process
• Variational Bayesian Linear Neural Networks

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →