Researchers have developed NVLinkOneSided, a novel one-sided expert-parallel all-to-all communication protocol designed for Mixture-of-Experts (MoE) inference on GPUs within a single NVLink domain. This method allows GPUs to push tokens directly into target buffers, bypassing metadata exchange and improving efficiency, especially at small batch sizes. Benchmarked on Blackwell GPUs, NVLinkOneSided demonstrated 1.2-3.0x faster dispatch and combine operations compared to DeepEP, and achieved up to 2.4x faster MoE layers, making it competitive even with fused backends like MegaMoE for decoding tasks. AI
IMPACT Accelerates Mixture-of-Experts model inference, potentially enabling faster and more efficient deployment of large AI models.
RANK_REASON Paper describing a new technical method for AI inference. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →