MoE models misroute tokens on complex reasoning tasks, study finds

Researchers have identified a significant issue in Mixture-of-Experts (MoE) language models where the routing mechanism, which directs tokens to specific experts, often selects suboptimal paths. While the standard router performs well on confident tokens, it fails to identify better-performing routes for complex reasoning tasks. This misrouting is present across several prominent MoE models, including Qwen3, GPT-OSS, DeepSeek-V2, and OLMoE. The study suggests that even a minor update to the router, without altering the experts themselves, can improve performance on challenging math and reasoning benchmarks, indicating that routing efficiency is a key bottleneck. AI

IMPACT Identifies a critical flaw in MoE routing that hinders reasoning capabilities, suggesting targeted router improvements could boost performance on complex tasks.

RANK_REASON Academic paper detailing a novel analysis of MoE model routing mechanisms and their impact on performance. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• DeepSeek-V2-Lite
• GPT-OSS-20B
• HMMT
• Mixture-of-Experts
• OLMoE-1B-7B
• Qwen3-30B-A3B

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →