PulseAugur
中
实时 12:38:02
English(EN) ShellElastic: Scalable Elastic Pretraining with Simultaneous Maximal Updates and Hyperparameter Transfer

ShellElastic 为基础模型提供可扩展的弹性预训练

作者 PulseAugur 编辑部 · [2 个来源] ·

研究人员开发了 ShellElastic，一种新颖的基础模型训练方法，可高效支持多样化的推理预算。该方法在单个超网内训练嵌套的子网络，克服了可变工作负载、有限的超参数迁移以及确保所有子网络同时更新等挑战。ShellElastic 将隐藏维度组织成一个密集核心和连续的层，从而实现固定的计算图和成本效益高的训练。该系统还引入了 \(\\mu\)E，这是最大更新参数化的一个扩展，它保证所有子网络都得到最大程度的更新，并跨不同模型尺寸保持最优超参数。 AI

影响 能够更有效地训练各种推理预算的基础模型，可能降低计算成本和开发时间。

排序理由 该集群描述了一篇详细介绍基础模型新训练方法的学术论文。

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

ShellElastic 为基础模型提供可扩展的弹性预训练

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
该集群描述了一篇详细介绍基础模型新训练方法的学术论文。
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
16 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [2]

  1. arXiv cs.AI TIER_1 English(EN) · Can Jin, Yan Kang, Yifan Gong, Chongjian Ge, Mingze Xu, Yuanjun Xiong, Dimitris N. Metaxas, Hongwu Peng ·

    ShellElastic: 可扩展的弹性预训练，同时实现最大化更新和超参数迁移

    arXiv:2610.15124v1 Announce Type: cross Abstract: Foundation models need to support diverse inference budgets, yet training a separate model for each budget duplicates pretraining effort. Elastic models train several nested sub-networks inside one weight-shared supernet, but exis…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    ShellElastic：可扩展的弹性预训练，同时实现最大更新和超参数迁移

    Foundation models need to support diverse inference budgets, yet training a separate model for each budget duplicates pretraining effort. Elastic models train several nested sub-networks inside one weight-shared supernet, but existing recipes face three obstacles: variable per-st…

相关实体

相关话题