研究人员开发了EcoSpec，一个新颖的成本感知推测解码框架，旨在提高混合专家（MoE）大语言模型的推理效率。该方法通过将预测的边际专家激活成本纳入草稿选择过程，解决了“专家分散”的问题，旨在重用专家并减少内存流量。在DeepSeek-V3.1和Qwen3-235B-A22B等大规模MoE模型上的评估表明，EcoSpec持续减少了活跃专家足迹，并将解码速度提高了1.62倍。 AI
影响 这项研究可能导致更高效的部署和更快的混合专家模型推理，从而可能降低运营成本。
排序理由 该集群包含一篇详细介绍改进大语言模型推理效率的新方法的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2607.12696v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) models have become an important approach for scaling Large Language Models (LLMs), but their inference efficiency depends strongly on expert activation patterns. Speculative decoding (SD) accelerate…
Sparse Mixture-of-Experts (MoE) models have become an important approach for scaling Large Language Models (LLMs), but their inference efficiency depends strongly on expert activation patterns. Speculative decoding (SD) accelerates autoregressive generation by verifying multiple …