English(EN) The Spectral Dynamics and Noise Geometry of Muon

新的“μ子”优化技术使矩阵梯度趋于平缓

作者 PulseAugur 编辑部 · [2 个来源] · 2026-06-07 00:51

一篇新研究论文介绍了一种名为“μ子”（Muon）的优化技术，该技术用极向因子取代矩阵梯度。此方法保持了奇异方向，但使更新谱趋于平缓，作者认为这在某些训练模式下可能是有益的。实验表明，与AdamW相比，μ子可以在小规模NanoGPT预训练中改善验证损失，尽管其有效性取决于具体的训练模式。 AI

影响这种新的优化方法可能为AdamW等标准优化器提供一种替代方案，有可能在特定的模型架构中提高训练稳定性和性能。

排序理由该集群包含一篇详细介绍机器学习模型新优化技术的学术论文。

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Research; 该集群包含一篇详细介绍机器学习模型新优化技术的学术论文。
Source corroboration: 2 independent sources; Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics: paper, other; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 123 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

arXiv stat.ML TIER_1 English(EN) · Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso Poggio · 2026-06-09 04:00

μ子的谱动力学与噪声几何

arXiv:2606.08388v1 Announce Type: cross Abstract: Muon replaces a matrix gradient $G=U\Sigma V^\top$ by its polar factor $UV^\top$. This keeps the singular directions selected by the gradient, but makes the update spectrum flat. We study the optimization bias created by this oper…
arXiv stat.ML TIER_1 English(EN) · Tomaso Poggio · 2026-06-07 00:51

μ子的谱动力学与噪声几何

Muon replaces a matrix gradient $G=UΣV^\top$ by its polar factor $UV^\top$. This keeps the singular directions selected by the gradient, but makes the update spectrum flat. We study the optimization bias created by this operation. Under explicit alignment assumptions, we prove th…