研究人员开发了 Ringmaster LMO，一种新颖的异步神经网络训练方法，解决了分布式系统中的效率低下问题。该方法基于延迟阈值概念来管理梯度陈旧性，旨在提高异构环境下的训练速度。该方法专为无约束随机非凸优化设计，并在涉及二次问题和语言模型预训练的实验中，与现有的同步和异步基线相比，表现出卓越的性能。 AI
影响 这种异步优化方法可以加速分布式和异构计算环境中的大规模模型训练。
排序理由 该集群包含一篇详细介绍机器学习优化新方法的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.18174v1 Announce Type: cross Abstract: Muon has recently emerged as a strong alternative to AdamW for training neural networks, with encouraging large-scale pretraining results and growing evidence that matrix-structured updates can be faster in practice. Yet Muon, and…
Muon has recently emerged as a strong alternative to AdamW for training neural networks, with encouraging large-scale pretraining results and growing evidence that matrix-structured updates can be faster in practice. Yet Muon, and more generally Linear Minimization Oracle (LMO) b…