Researchers have developed a novel method to recover curvature information from gradients alone, a fundamental challenge in machine learning. Their technique involves injecting zero-mean noise into targets, with variance calibrated to the local output curvature. This approach allows for the recovery of practical curvature matrices, such as the Generalised Gauss--Newton (GGN) matrix for non-linear models and diagonal blocks of the population Hessian for ReLU networks, without needing per-sample gradients or second-order derivatives. The method is demonstrated to be simple, lightweight, provably optimal, and easily integrated into existing training pipelines, showing improved optimization and utility for early stopping in experiments. AI
IMPACT Enables more efficient optimization and sampling in machine learning models by recovering curvature information from gradients.
RANK_REASON Academic paper detailing a new method for machine learning optimization. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- Calibrated Target Noise
- CatalyzeX
- DagsHub
- Generalised Gauss--Newton
- Gotit.pub
- Hugging Face
- IArxiv
- rectifier
- ScienceCast
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
