English(EN) STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules

STAPLE方法通过静态稀疏内存增强LLM容量

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-12 04:00

研究人员开发了STAPLE，一种将静态稀疏内存集成到预训练语言模型中的新颖方法。该方法在不按比例增加计算成本的情况下增强了模型容量，性能优于现有的稀疏升级技术。STAPLE在HumanEval和GSM8K等基准测试中取得了显著的性能提升，并且与Mixture-of-Experts模型相比，在速度、内存使用和能耗方面也表现出更高的效率。 AI

影响该方法提供了一种更有效、更具可扩展性的扩展预训练模型的方法，有可能降低开发更大LLM的成本和能耗。

排序理由该集群包含一篇详细介绍LLM升级新方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 11 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍LLM升级新方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Same-day; Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Ranajoy Sadhukhan, Euijin Hong, Sheng Cao, Zechun Liu, Shuowei Jin, Sha Li, Changlong Yu, Xin Liu, Qin Lu, Zhenyu Shi, Priyanka Nigam, Beidi Chen · 2026-10-12 04:00

STAPLE：通过嵌入模块对预训练检查点进行静态稀疏升级

arXiv:2610.12599v1 Announce Type: new Abstract: Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures remains expensive and data-hungry. Existing sparse upcycling methods mainly targ…

本文如何被排名

报道来源 [1]

STAPLE：通过嵌入模块对预训练检查点进行静态稀疏升级

相关实体

相关话题