English(EN) Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism

Piper框架通过资源建模提升MoE模型训练效率

作者 PulseAugur 编辑部 · [2 个来源] · 2026-05-06 15:47

一个名为Piper的新框架已被开发出来，用于解决在高性n能计算（HPC）平台上训练大型混合专家（MoE）模型所面临的挑战。Piper利用资源建模来优化训练策略，侧重于流水线并行和高效通信。这种方法旨在克服MoE架构固有的内存占用大、通信瓶颈和工作负载不平衡等问题。 AI

影响引入了一个框架，显著提高了训练大型MoE模型的效率和可扩展性，有望降低成本并加速前沿模型的开发。

排序理由这是一篇详细介绍用于高效大规模MoE训练的新框架的研究论文。

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

报道来源 [2]

arXiv cs.LG TIER_1 English(EN) · Sajal Dash, Feiyi Wang · 2026-05-07 04:00

Piper：通过资源建模和流水线混合并行实现高效大规模 MoE 训练

arXiv:2605.05049v1 Announce Type: cross Abstract: Frontier models increasingly adopt Mixture-of-Experts (MoE) architectures to achieve large-model performance at reduced cost. However, training MoE models on HPC platforms is hindered by large memory footprints, frequent large-sca…
arXiv cs.AI TIER_1 English(EN) · Feiyi Wang · 2026-05-06 15:47

Piper：通过资源建模和流水线混合并行实现高效大规模 MoE 训练

Frontier models increasingly adopt Mixture-of-Experts (MoE) architectures to achieve large-model performance at reduced cost. However, training MoE models on HPC platforms is hindered by large memory footprints, frequent large-scale communication across heterogeneous networks, an…