研究人员推出MobileMoE，这是一系列专为移动部署设计的新型端侧专家混合（MoE）语言模型。这些模型拥有不足十亿的激活参数，通过优化MoE架构以适应移动设备的内存和计算限制，为端侧LLM树立了新的性能标杆。与领先的密集LLM和现有的MoE模型相比，MobileMoE模型在14项基准测试中表现出竞争力或更优的性能，同时使用的FLOPs和参数显著减少。该项目还实现了智能手机上首个高效MoE推理，展示了预填充和解码时间的显著加速。 AI
影响 为端侧LLM树立了新的帕累托前沿，可能加速先进AI能力在移动设备上的部署。
排序理由 该集群包含一篇详细介绍新模型架构及其性能基准的学术论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
arXiv:2605.27358v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) has become the de facto architecture for hundred-billion-parameter language models, yet its advantages at sub-billion scales for on-device deployment remain largely unexplored. To close this gap, we presen…
Mixture-of-Experts (MoE) has become the de facto architecture for hundred-billion-parameter language models, yet its advantages at sub-billion scales for on-device deployment remain largely unexplored. To close this gap, we present MobileMoE, a family of on-device MoE language mo…
Mixture-of-Experts (MoE) has become the de facto architecture for hundred-billion-parameter language models, yet its advantages at sub-billion scales for on-device deployment remain largely unexplored. To close this gap, we present MobileMoE, a family of on-device MoE language mo…
MobileMoE introduces efficient on-device Mixture-of-Experts language models with sub-billion parameters that achieve better performance and efficiency compared to dense baselines and existing MoE models.