English(EN) DivMoE: Fine-Grained MoE Upcycling via Cross-Domain Expert Composition

DivMoE框架能够实现高效的细粒度MoE模型升级

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-09 04:00

研究人员开发了DivMoE，一个用于从预训练的密集模型高效创建专家混合（MoE）模型的新框架。先前的方法在细粒度升级方面存在困难，导致准确率下降。DivMoE通过使用领域专业化专家初始化和多样性约束路由来解决这个问题，确保更好的性能并避免准确率回归。该框架已显示出具有竞争力的结果，在准确率方面可与更大的模型相媲美，同时使用的参数更少。 AI

影响通过升级现有的密集模型，能够更高效地创建强大的AI模型，从而可能降低先进MoE架构的入门门槛。

排序理由该集群包含一篇详细介绍新AI模型创建方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 17 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍新AI模型创建方法的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Yuxuan Lou, Kai Yang, Geng Zhang, Yong Liu, Yang You · 2026-10-09 04:00

DivMoE：通过跨域专家组合实现细粒度MoE升级

arXiv:2610.11317v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures have become essential for scaling large language models, with recent work demonstrating the benefits of fine-grained expert designs. Training such models from scratch is expensive, and sparse u…

本文如何被排名

报道来源 [1]

DivMoE：通过跨域专家组合实现细粒度MoE升级

相关实体

相关话题