两篇新的研究论文探讨了深度学习中在稳定性边缘（EoS）运行的梯度下降现象。第一篇论文介绍了“Edge Flow”，这是一个微分方程系统，用于模拟EoS下的梯度下降动力学，将其分解为中心、振荡方向和幅度。第二篇论文提出了一个适用于过参数化神经网络的分岔理论框架，展示了稳定的EoS训练如何从翻转分岔产生，并在特定条件下证明了收敛到最小化流形。 AI
影响 这些框架为理解和潜在地改进深度学习训练过程的稳定性和收敛性提供了新的理论工具。
排序理由 两篇在arXiv上发表的学术论文，提出了新的深度学习动力学理解理论框架。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2606.18080v1 Announce Type: new Abstract: Gradient descent in deep learning may operate at the edge of stability (EoS), a regime in which the largest eigenvalue of the loss Hessian hovers near the stability threshold $2/\eta$, where $\eta$ is the learning rate. Classical an…
Gradient descent in deep learning may operate at the edge of stability (EoS), a regime in which the largest eigenvalue of the loss Hessian hovers near the stability threshold $2/η$, where $η$ is the learning rate. Classical analysis tools such as gradient flow and the descent lem…
arXiv:2606.15551v1 Announce Type: new Abstract: The Edge of Stability (EoS) phenomenon, where gradient descent operates with sharpness exceeding the classical convergence threshold yet the loss decreases over long timescales, is ubiquitous in modern deep learning but remains poor…