English(EN) SLEM: Spectrum-Aware Latent Expert Merging

新的SLEM框架降低了MoE模型的内存开销

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-15 04:00

研究人员开发了频谱感知潜在专家合并（SLEM）框架，该框架旨在降低混合专家（MoE）模型的内存开销。与直接插值参数或依赖校准数据的先前方法不同，SLEM在源自变分自编码器的潜在空间中运行。该方法根据潜在相似性对专家进行分组，并使用它们的权重频谱来确定合并系数，从而实现几何感知插值。SLEM在通用知识基准测试中已显示出相当或更优的性能，并在涉及数学推理和代码生成任务方面取得了改进的结果。 AI

影响该方法可能导致更节省内存的大型语言模型，从而降低部署成本并提高可访问性。

排序理由该条目是一篇研究论文，详细介绍了一种改进AI模型架构的新技术方法。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 14 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该条目是一篇研究论文，详细介绍了一种改进AI模型架构的新技术方法。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Breaking (< 6h); Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Dohyeon Kim, Bedionita Soro, Bruno Andreis, Sung Ju Hwang · 2026-10-15 04:00

SLEM：感知频谱的潜在专家合并

arXiv:2610.16484v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale model capacity efficiently by activating only a small subset of experts for each input, while still requiring all parameters to be stored in memory, resulting in substantial memory overhead. E…

本文如何被排名

报道来源 [1]

SLEM：感知频谱的潜在专家合并

相关实体

相关话题