一篇新的研究论文为理解 Muon 和 Scion 等非欧几里得优化方法在训练 Transformer 模型中的成功提供了理论框架。该研究侧重于重尾非凸区域，证明这些方法通过吸收噪声而不像其欧几里得对应物那样依赖于维度，从而实现了最优样本复杂度。研究结果得到了大型语言模型实验的支持，并表明其他 Schatten 几何形状具有竞争力的潜力。 AI
影响 为训练大型语言模型所使用的先进优化技术提供了理论依据。
排序理由 该集群包含一篇详细介绍机器学习优化方法理论进展的研究论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.14560v1 Announce Type: cross Abstract: Non-Euclidean optimisation methods with matrix-valued updates, such as Muon and Scion, have recently shown strong empirical performance for training Transformer models, yet their theoretical advantages over Euclidean methods remai…
Non-Euclidean optimisation methods with matrix-valued updates, such as Muon and Scion, have recently shown strong empirical performance for training Transformer models, yet their theoretical advantages over Euclidean methods remain poorly understood. We address this gap in the he…