English(EN) Understanding Stability of Outer Nesterov Momentum on Stochastic Pseudo-Gradients

Nesterov 加速动量在大规模神经网络上的稳定性分析

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-19 04:00

研究人员分析了 Nesterov 加速动量应用于大规模神经网络中跨多个工作节点训练的随机伪梯度时的稳定性。他们推导出了随机二次函数的均方稳定性判据，表明内层批量大小决定了更多局部步骤是导致稳定性还是不稳定性。研究还发现，虽然平均伪梯度可以允许更高的外层动量，但稳定性所需的工作节点数量可能随局部步骤呈指数级增长。这些动态可以通过在神经网络中间检查点使用共享小批量扰动来探测。 AI

影响为优化大规模神经网络提供了理论见解，可能提高训练效率和稳定性。

排序理由学术论文，详细介绍了分布式训练中优化稳定性的理论分析和实验结果。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 15 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 学术论文，详细介绍了分布式训练中优化稳定性的理论分析和实验结果。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Nishant Sharma, Laura Gomezjurado Gonzalez, Ganesh Talluri, Sarath Chandar, Hiroki Naganuma · 2026-10-19 04:00

理解外层Nesterov动量在随机伪梯度上的稳定性

arXiv:2610.22623v1 Announce Type: new Abstract: Large neural networks are commonly trained across many workers, and communication between them is a major cost. Workers therefore take several local optimization steps before sending their net parameter displacements, or pseudo-grad…

本文如何被排名

报道来源 [1]

理解外层Nesterov动量在随机伪梯度上的稳定性

相关实体

相关话题