一篇新论文分析了神经网络中的过参数化如何通过引入额外的对称性来帮助优化。这些对称性充当了Hessian的预条件形式，从而导致更好的条件最小化。此外，过参数化增加了找到接近典型初始化的全局最小值的可能性，使其更容易获得。通过师生网络进行的实验证实了这些理论预测，显示随着网络宽度的增加，收敛性和条件数都有所改善。 AI
影响 为理解网络宽度如何影响优化和收敛提供了理论框架。
排序理由 关于神经网络优化理论方面的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2604.25150v1 Announce Type: new Abstract: Overparameterization is central to the success of deep learning, yet the mechanisms by which it improves optimization remain incompletely understood. We analyze weight-space symmetries in neural networks and show that overparameteri…
Overparameterization is central to the success of deep learning, yet the mechanisms by which it improves optimization remain incompletely understood. We analyze weight-space symmetries in neural networks and show that overparameterization introduces additional symmetries that ben…