一篇新论文研究了权重衰减在深度学习训练稳定性中的作用，挑战了其作为简单正则化技术的普遍看法。研究分析了权重衰减如何影响“稳定性边缘”的参数动态和损失锐度，证明它能有效减缓渐进锐化。研究还揭示了一种依赖于架构的相变，其中权重衰减在CNN中会抑制振荡，但在MLP中，由于参数向量和锐度梯度的对齐，它会在理论边界以下稳定锐度。 AI
影响 研究了训练稳定性的基本机制，可能带来更强大、更高效的深度学习模型开发。
排序理由 这是一篇发表在arXiv上的研究论文，详细介绍了关于机器学习技术的新发现。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.16622v1 Announce Type: cross Abstract: In modern deep learning, weight decay is often credited with "stabilizing" training dynamics, diverging from its classical role as a static regularization penalty. We investigate a fundamental question: *does weight decay stabiliz…
In modern deep learning, weight decay is often credited with "stabilizing" training dynamics, diverging from its classical role as a static regularization penalty. We investigate a fundamental question: *does weight decay stabilize training dynamics, and if so, through which mech…