新的EGGROLL方法通过低秩演化策略增强LLM训练

作者 PulseAugur 编辑部 · [1 个来源] · 2026-09-11 04:00

研究人员开发了EGGROLL方法，通过使用低秩高斯乘积代替密集权重扰动，使演化策略（ES）能够应用于大型语言模型（LLM）。虽然这种方法计算效率高，但存在几何限制。该研究分析了更新场和估计误差，表明EGGROLL对于二次目标是精确的，并为平滑目标提供了界限。引入了一种新的留一法估计器LOO-ROLL，以保持有限秩种群场，提高了Transformer块的准确性，并显著提升了高达80亿参数模型的GSM8K测试集的性能。 AI

影响这项研究可能导致更高效的大型语言模型训练，从而提高它们在复杂推理任务上的性能。

排序理由该集群包含一篇详细介绍LLM训练新方法的论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 15 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍LLM训练新方法的论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Ege C. Kaya, Abolfazl Hashemi · 2026-09-11 04:00

EGGROLL, Unrolled: 理解和改进大规模低秩演化策略

arXiv:2609.10980v1 Announce Type: new Abstract: EGGROLL makes evolution strategies (ES) practical for LLMs by replacing dense Gaussian weight perturbations with low-rank Gaussian products, often of rank one. This choice is computationally attractive but geometrically severe: each…

本文如何被排名

报道来源 [1]

EGGROLL, Unrolled: 理解和改进大规模低秩演化策略

相关实体

相关话题