一篇新论文分析了用于混合专家（MoE）大型语言模型（LLM）服务的网络拓扑，发现低成本的无交换机网络比昂贵的横向扩展基础设施更具成本效益。研究表明，降低当前横向扩展网络中的链路带宽可以将成本效益提高高达27%。研究建议，无交换机拓扑，特别是3D全网状结构，提供了更优的性能-成本权衡，并且这种优势有望随着未来GPU的迭代而持续。 AI
影响 通过优化网络拓扑，为LLM服务基础设施节省大量成本。
排序理由 分析LLM服务基础设施的学术论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2605.00254v1 Announce Type: cross Abstract: Mixture-of-experts (MoE) architectures have turned LLM serving into a cluster-scale workload in which communication consumes a considerable portion of LLM serving runtime. This has prompted industry to invest heavily in expensive …
Mixture-of-experts (MoE) architectures have turned LLM serving into a cluster-scale workload in which communication consumes a considerable portion of LLM serving runtime. This has prompted industry to invest heavily in expensive high-bandwidth scale-up networks. We question whet…