两篇新研究论文探讨了更高效训练大型语言模型（LLM）的方法。第一篇论文《关于LLM训练学习率缩放的非线性研究》调查了当前学习率外推法的局限性，并提出最优学习率在更大规模下呈现向上曲率，这可以通过关注有效学习率或数据外推来缓解。第二篇论文《面向高效LLM训练的几何原理随机优化》引入了新算法GrassWalk和GrassJump，它们利用梯度子空间的几何特性来提高优化效率，在LLaMA和Qwen等模型上取得了最先进的成果。 AI
影响 这些论文提供了新的理论和算法方法来降低训练大型语言模型的计算成本，可能加速其开发和部署。
排序理由 两篇发表在arXiv上的学术论文，详细介绍了提高LLM训练效率的新方法。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
arXiv:2607.01678v1 Announce Type: new Abstract: Communication increasingly dominates the cost of Large Language Model (LLM) pre-training, especially under data-parallel and sharded training schemes, where gradient synchronization and parameter reconstruction overhead increase wit…
arXiv:2606.29158v1 Announce Type: cross Abstract: Learning-rate transfer can reduce the cost of training large language models: instead of sweeping learning rates at target scale, practitioners extrapolate from smaller runs. Existing approaches often assume that the optimal learn…
arXiv:2510.01878v2 Announce Type: replace Abstract: Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches employed primarily for computational efficiency.…