English(EN) OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling

OrScale优化方法改进神经网络训练

作者 PulseAugur 编辑部 · [1 个来源] · 2026-05-08 14:47

研究人员推出了一种名为OrScale的新型优化技术，旨在增强神经网络训练。OrScale在Muon方法的基础上，通过引入层级信任比率缩放来衡量实际应用参数空间方向的Frobenius范数。这一新论文中详细介绍的方法旨在改进现有的Muon和AdamW等方法，尤其适用于语言模型。 AI

影响引入了一种新的优化技术，在基准测试中显示出实证改进，有望提高模型训练效率。

排序理由该集群包含一篇详细介绍新研究方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍新研究方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, other; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 168 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

arXiv cs.CL TIER_1 English(EN) · Yang You · 2026-05-08 14:47

OrScale：逐层信任比率缩放的正交优化

Muon improves neural-network training by orthogonalizing matrix-valued updates, but it leaves each layer's update magnitude controlled mostly by a global learning rate. We introduce OrScale, a trust-ratio extension of Muon built on a simple rule: the denominator of a layer-wise r…