SHAPE framework prunes MoE LLMs by modeling expert coalitions

Researchers have developed a new framework called SHAPE for pruning experts in sparse Mixture-of-Experts (MoE) large language models. Unlike previous methods that evaluated experts independently, SHAPE considers the cooperative nature of MoE inference, where experts work in coalitions. The framework uses a Shapley-style attribution to identify experts crucial for high-utility collaborations, leading to more effective pruning. Experiments on models like Qwen3-30B-A3B, GPT-OSS-20B, and DeepSeek-V2-Lite demonstrated that SHAPE can significantly reduce memory footprint without substantial accuracy loss, even with up to 40% expert pruning. AI

IMPACT Enables more efficient deployment of large MoE models by reducing memory requirements without sacrificing accuracy.

RANK_REASON The cluster contains a research paper detailing a new method for pruning MoE LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• DeepSeek-V2-Lite
• GPT-OSS-20B
• Mixture-of-Experts
• Qwen3-30B-A3B

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →