Distributional Coverage Expert Pruning

New pruning methods reduce MoE LLM memory footprint while preserving performance

Researchers have developed new methods for pruning experts in Mixture-of-Experts (MoE) large language models, addressing the significant memory overhead associated with storing all experts. The proposed techniques, Proj…