PulseAugur
EN
LIVE 19:04:39

New NVLinkOneSided protocol accelerates MoE inference on GPUs

Researchers have developed NVLinkOneSided, a novel one-sided expert-parallel all-to-all communication protocol designed for Mixture-of-Experts (MoE) inference on GPUs within a single NVLink domain. This method allows GPUs to push tokens directly into target buffers, bypassing metadata exchange and improving efficiency, especially at small batch sizes. Benchmarked on Blackwell GPUs, NVLinkOneSided demonstrated 1.2-3.0x faster dispatch and combine operations compared to DeepEP, and achieved up to 2.4x faster MoE layers, making it competitive even with fused backends like MegaMoE for decoding tasks. AI

IMPACT Accelerates Mixture-of-Experts model inference, potentially enabling faster and more efficient deployment of large AI models.

RANK_REASON Paper describing a new technical method for AI inference. [lever_c_demoted from research: ic=1 ai=1.0]

Read on Hugging Face Daily Papers →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

New NVLinkOneSided protocol accelerates MoE inference on GPUs

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
Paper describing a new technical method for AI inference. [lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
infra, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
12 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

Full methodology in our editorial standards.

COVERAGE [1]

  1. Hugging Face Daily Papers TIER_1 English(EN) ·

    One-Sided Expert-Parallel All-to-All for Mixture-of-Experts Inference over NVLink

    Expert parallelism (EP) shards the experts of a Mixture-of-Experts (MoE) layer across GPUs, so every MoE layer requires two all-to-all communications, namely dispatch and combine. Compared with training, inference processes relatively few tokens per GPU at a time, and at small ba…