English(EN) One-Sided Expert-Parallel All-to-All for Mixture-of-Experts Inference over NVLink

新的NVLink技术加速混合专家AI推理

作者 PulseAugur 编辑部 · [1 个来源] · 2026-11-11 05:00

研究人员开发了一种名为NVLinkOneSided的新方法，用于加速NVLink上的混合专家（MoE）模型的推理。该方法通过允许GPU之间直接推送Token来优化专家并行全互联通信，绕过元数据交换并降低延迟，尤其适用于小批量大小。在Blackwell GPU上的基准测试显示，NVLinkOneSided相比DeepEP等现有方法实现了1.2-3.0倍的加速，使MoE层速度提升高达2.4倍，在解码任务中具有竞争力。 AI

影响优化MoE推理，可能降低大型语言模型的延迟并提高吞吐量。

排序理由详细介绍MoE模型新推理优化技术的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 33 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 详细介绍MoE模型新推理优化技术的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: infra, paper; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CL TIER_1 English(EN) · Bo Li, Chulian Zhang, Or Biri, Anthony Chang, Anton Korzh, Noam Duvdevani, XingFei Xi, Daniel Stokes, Zhenhuan Chen, Ludwig Schneider, Artem Y. Polyakov, Julien Demouth, Chandler Zhou · 2026-11-11 05:00

面向NVLink上混合专家模型推理的单边专家并行全互联

arXiv:2611.13188v1 Announce Type: cross Abstract: Expert parallelism (EP) shards the experts of a Mixture-of-Experts (MoE) layer across GPUs, so every MoE layer requires two all-to-all communications, namely dispatch and combine. Compared with training, inference processes relati…

本文如何被排名

报道来源 [1]

面向NVLink上混合专家模型推理的单边专家并行全互联

相关实体

相关话题