新的路由方法加速大型MoE模型

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-27 04:00

研究人员推出了一种名为Prefetch-Steered Routing (PSR) 的新方法，用于优化无法完全载入GPU内存的混合专家（MoE）模型的解码过程。PSR基于逻辑缓存确定性地路由专家，减少了从主机内存复制权重的昂贵开销。该方法展示了显著的加速效果，在OLMoE-1B-7B上解码速度比精确路由快2.2-2.3倍，并在Moonlight-16B-A3B和Qwen3.6-35B-A3B的基准测试中有所改进，具体取决于工作负载和驻留情况。 AI

影响该方法可以显著提高大型MoE模型的部署效率和速度，使其在内存有限的硬件上更易于使用。

排序理由该集群包含一篇详细介绍AI模型性能优化新技术的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 12 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍AI模型性能优化新技术的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Same-day; Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Siming Huang · 2026-10-27 04:00

预取引导路由：用于卸载式专家混合解码的确定性感知缓存专家选择

arXiv:2610.34721v1 Announce Type: new Abstract: When a Mixture-of-Experts (MoE) model does not fit in GPU memory, batch-1 decoding is dominated by copying expert weights from host memory. Cache-aware routing reduces these copies by biasing the router toward resident experts, but …

本文如何被排名

报道来源 [1]

预取引导路由：用于卸载式专家混合解码的确定性感知缓存专家选择

相关实体

相关话题