Researchers pinpoint origin of neural network 'Edge of Stability' phenomenon

Researchers have introduced a new concept called the 'edge coupling' to explain the phenomenon known as the Edge of Stability in neural network training. This functional, applied to consecutive iterate pairs, helps to explain why the largest Hessian eigenvalue is driven to the threshold of $2/\eta$ (where $\eta$ is the learning rate) during full-batch gradient descent. The proposed method provides an exact forcing of the Hessian eigenvalue without any gap, offering a more unified explanation for this observed behavior. AI

IMPACT Provides a theoretical framework that could lead to more stable and efficient neural network training.

RANK_REASON Academic paper detailing a new theoretical explanation for a phenomenon in neural network training.

Read on arXiv stat.ML →

• arXiv
• Edge of Stability
• gradient descent
• Hessian eigenvalue
• learning rate
• neural networks

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →