New research details multi-pass SGD learning dynamics in high-dimensional models

A new paper published on arXiv details the learning dynamics of multi-pass Stochastic Gradient Descent (SGD) in high-dimensional, non-convex multi-index models. The research develops an asymptotically exact characterization of these dynamics, revealing a continuous-time scaling limit governed by dynamical mean-field equations driven by a Poisson jump process. This framework connects various existing theories of SGD and suggests that multi-pass SGD may converge to local optima with distinct statistical properties compared to diffusion approximations or gradient flow. AI

IMPACT Provides a theoretical framework for understanding SGD optimization in complex models, potentially influencing future algorithm design.

RANK_REASON Academic paper published on arXiv detailing theoretical advancements in machine learning dynamics. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• DagsHub
• dynamical mean-field equations
• Gotit.pub
• Gradient Flow
• Hugging Face
• Least Squares Problems with Absolute Quadratic Constraints
• Leda Wang
• multi-index model
• Poisson jump process
• SGD
• stochastic gradient descent

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →