New framework enhances MoE LLMs on noisy analog hardware

Researchers have introduced ROMER, a post-training calibration framework designed to enhance the robustness of Mixture-of-Experts (MoE) Large Language Models (LLMs) when deployed on analog Compute-in-Memory (CIM) systems. This framework addresses hardware imperfections in CIM by replacing underutilized experts and recalibrating router decisions to maintain load balance and optimal routing under noisy conditions. Experiments show ROMER significantly reduces perplexity for models like DeepSeek-MoE, Qwen-MoE, and OLMoE when subjected to real-chip noise. AI

影响 Improves the viability of deploying LLMs on energy-efficient analog hardware by mitigating noise-induced performance degradation.

排序理由 The cluster contains an academic paper detailing a new method for improving LLM performance on specific hardware. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

• CIM
• DeepSeek-MoE
• LLMs
• Qwen-MoE
• ROMER
• OLMoE

• 论文
• 基础设施

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →