新的FOAM优化器以一半的状态匹配SOAP/Muon LLM训练增益

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-12 00:00

研究人员推出了一种新的SOAP优化器变体FOAM，专为大型语言模型预训练而设计。FOAM旨在实现类似SOAP和Muon等二阶优化器的性能优势，同时显著降低计算和内存开销。它通过使用仅输入预处理和因子化Adam-moment估计器来实现这一点，这使优化器状态减半，并将训练时间缩短至与Muon类模型相当。 AI

影响 FOAM的效率提升可以使在计算资源减少的情况下进行更大规模的LLM预训练。

排序理由该集群包含一篇详细介绍LLM预训练新优化方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 1 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍LLM预训练新优化方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 1 days old; Coverage has settled into its steady-state source set.

完整方法见我们的编辑标准。

报道来源 [1]

Hugging Face Daily Papers TIER_1 English(EN) · 2026-11-12 00:00

面向实际和可扩展SOAP的因子化单边Adam-Moment

FOAM is a lightweight SOAP variant that uses input-only preconditioning and a factored Adam-moment estimator to match strong second-order and Muon baselines in LLM pre-training while using half the optimizer state and running at Muon-class speed.

本文如何被排名

报道来源 [1]

面向实际和可扩展SOAP的因子化单边Adam-Moment

相关实体

相关话题