PulseAugur
中
实时 06:56:50

Muon 优化器在 LLM 训练中实现新的迭代复杂度

一种名为 Muon 的新优化算法在进行置乱(shuffling)时,已证明在光滑、非凸有限和问题上找到 $\varepsilon$-平稳点的迭代复杂度为 $\mathcal{O}(\varepsilon^{-3})$。此复杂度与置乱下的随机梯度下降算法相当,并优于 Muon 此前在有放回抽样下的 $\mathcal{O}(\varepsilon^{-4})$ 复杂度。研究还指出了 Muon 在固定动量系数 $\beta$ 下的一个失效案例,表明在某些问题配置下,它可能无法收敛到平稳点,这提示了在训练损失停滞时,增加 $\beta$ 和减小步长的实际调优规则。 AI

影响 这项研究通过改进优化算法,有望实现更高效的大型语言模型训练。

排序理由 学术论文,详细介绍了新的优化算法及其理论复杂度。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

Muon 优化器在 LLM 训练中实现新的迭代复杂度

本文如何被排名

Signal score
38 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
学术论文,详细介绍了新的优化算法及其理论复杂度。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

  1. arXiv cs.LG TIER_1 English(EN) · Kun Huang, Shi Pu, Karl Henrik Johansson ·

    Muon Attains $\mathcal{O}(\varepsilon^{-3})$ Iteration Complexity under Shuffling

    arXiv:2611.18124v1 Announce Type: cross Abstract: Muon is a stochastic first-order optimizer for matrix-valued parameters. It maintains an exponential moving average of stochastic gradients in a momentum buffer with coefficient $\beta$, and orthogonalizes the buffer to obtain the…