一篇新的研究论文探讨了 Muon 优化器在大型语言模型训练中优于 Adam 的性能优势。该研究题为“Muon 优于 Adam 的原因：曲率视角”，认为 Muon 通过承担较小的二阶曲率惩罚来实现更高的效率。这种优势归因于较低的归一化方向性锐度 (NDS)，而非更新尺度的差异，数据不平衡和层内曲率起着重要作用。 AI
影响 提供了对优化技术的更深入理解，可能带来更高效的 LLM 训练。
排序理由 该集群包含一篇学术论文，详细介绍了优化器性能的新视角。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.04662v1 Announce Type: cross Abstract: Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear. Our work takes a first step toward demystifying Muon's superiority …
Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear. Our work takes a first step toward demystifying Muon's superiority over Adam from a curvature perspective. First, we …