English(EN) ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models

ProQuant框架提升MoE模型量化质量

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-22 06:01

研究人员开发了ProQuant，一个用于混合专家（MoE）大语言模型训练后量化的新框架。该方法通过考虑跨层误差传播和缓解路由漂移，解决了在极低比特预算下保持生成质量的挑战。ProQuant采用剪枝或优先混合精度分配（PoP）策略进行联合专家剪枝和比特宽度分配，并结合路由一致性RMSNorm微调（RoCo）技术来适应专家表示。实验表明，ProQuant在各种生成任务的质量保持方面优于现有方法。 AI

影响这项研究可能导致大语言模型（特别是混合专家架构）更高效的存储和部署。

排序理由该集群描述了一篇详细介绍新型模型量化框架的新研究论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群描述了一篇详细介绍新型模型量化框架的新研究论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 15 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

Hugging Face Daily Papers TIER_1 English(EN) · 2026-10-22 06:01

ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models

Mixture-of-Experts (MoE) large language models reduce computation through sparse activation but still incur substantial storage costs. Mixed-precision post-training quantization (PTQ) reduces these costs by quantizing different model components at different bit widths, but existi…

本文如何被排名

报道来源 [1]

ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models

相关实体

相关话题