Peijun Zhuang

New framework tackles MoE LLM trilemma with dynamic clustering and compression

Researchers have developed a new framework to address the trilemma faced by Mixture-of-Experts (MoE) Large Language Models (LLMs), which involves load imbalance, parameter redundancy, and communication overhead. Their m…