一篇题为“SGD at the Edge of Stability: Stochastic Stabilization with Large Learning Rates”的新研究论文，探讨了随机梯度下降（SGD）在深度学习中的行为。该研究为SGD应用于多类交叉熵损失函数在线性分类器和两层神经网络时提供了理论收敛保证。研究表明，虽然SGD的随机性可能导致其在不稳定和稳定状态之间振荡，但该算法能够内在地实现自稳定，即使在较大的学习率下也能确保收敛。 AI
影响 为SGD的行为提供了理论见解，可能为深度学习模型的未来优化策略提供信息。
排序理由 该集群包含一篇发表在arXiv上的研究论文，详细介绍了机器学习优化方面的理论发现。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.30930v1 Announce Type: cross Abstract: Modern deep learning has been shown to operate at the edge of stability, routinely using learning rates far larger than those justified by classical optimization theory. Most prior analyses of the edge of stability phenomenon focu…
Modern deep learning has been shown to operate at the edge of stability, routinely using learning rates far larger than those justified by classical optimization theory. Most prior analyses of the edge of stability phenomenon focus on deterministic gradient descent, leaving the s…