English(EN) Hybrid Momentum Methods for Optimizing Deep Learning Models

新的混合动量优化器增强深度学习训练

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-21 04:00

研究人员开发了一种名为混合动量（HyM）的新型优化技术，用于深度学习模型。HyM 将过去梯度的线性加权平均与几何衰减的快速动量相结合，旨在更好地保留长期历史信息，同时适应近期变化。一种名为 HyMUON 的特定实现，结合了牛顿-舒尔茨正交化，在图像分类、机器翻译和语言建模任务中表现出优于现有方法的性能。值得注意的是，与 Muon 优化器相比，HyMUON 在 GPT-2 124M 和 LLaMA-1.3B 预训练中取得了更好的验证损失。 AI

影响引入了一种新颖的优化技术，可以提高大型语言模型和其他深度学习任务的训练效率和性能。

排序理由介绍深度学习模型新优化方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 介绍深度学习模型新优化方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 7 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Haifeng Xu, Yijin Ren, Qi Deng, Chenyu Zhou, Dongdong Ge, Yinyu Ye · 2026-10-21 04:00

混合动量法优化深度学习模型

arXiv:2610.27674v1 Announce Type: new Abstract: Momentum is central to stochastic optimization in deep learning, including the training of large language models. Classical momentum methods typically rely on a fast exponential moving average (EMA) scheme, which discounts past grad…

本文如何被排名

报道来源 [1]

混合动量法优化深度学习模型

相关实体

相关话题