New research explores expert deactivation in LLMs, revealing performance impacts

Researchers have investigated methods for deactivating experts in large language models during inference, finding that the approach significantly impacts performance. Their study compared four deactivation techniques—removal, freezing, remaining-expert renormalization, and unnormalized re-routing—across various instruction-tuned models. The findings indicate that retaining the training-time weight rule is crucial, as mismatches can lead to drastic increases in perplexity, with some cases exceeding the baseline by thousands of times. Specifically, on the OLMoE benchmark, strict-match accuracy dropped to zero across all tested deactivation methods. AI

IMPACT This research could inform more efficient inference strategies for large language models, potentially reducing computational costs and improving performance.

RANK_REASON The cluster contains a research paper published on arXiv detailing novel findings in LLM inference techniques. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• GSM8K
• Hugging Face
• Litmaps
• Olmoe
• ScienceCast
• scite Smart Citations

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →