Two new research papers explore the phenomenon of gradient descent operating at the edge of stability (EoS) in deep learning. The first paper introduces 'Edge Flow,' a system of differential equations that models gradient descent dynamics at EoS, decomposing them into center, oscillation direction, and magnitude. The second paper presents a bifurcation theory framework that applies to overparameterized neural networks, showing how stable EoS training arises from a flip bifurcation and proving convergence to the minimizing manifold under certain conditions. AI
IMPACT These frameworks offer new theoretical tools for understanding and potentially improving the stability and convergence of deep learning training processes.
RANK_REASON Two academic papers published on arXiv presenting new theoretical frameworks for understanding deep learning dynamics.
- bifurcation theory
- deep learning
- Edge of Stability
- generative adversarial network
- gradient descent
- Loss Landscape
- Neural Networks
- arXiv
- Hessian
AI-generated summary · Google Gemini · from 3 sources. How we write summaries →
