New framework quantifies uncertainty in neural network training dynamics

Researchers have developed a framework to analyze the fluctuations in training dynamics of wide neural networks. This work focuses on how these fluctuations, characterized by functional central limit theorems, propagate to nonlinear observables of the parameter distribution. The study applies a functional Delta method within weighted Sobolev spaces, enabling a central limit theorem for observables and providing a formula for their covariance. The findings offer a method for quantifying finite-width uncertainty and assessing whether specific observations contain the necessary information to identify meaningful quantities. AI

IMPACT Provides a theoretical framework for understanding and quantifying uncertainty in neural network training, potentially improving model robustness and interpretability.

RANK_REASON Academic paper detailing a new theoretical framework for analyzing neural network training dynamics. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Geoffrey Lacour
• Hugging Face
• Mean Field Neural Network Training
• stochastic gradient descent

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →