Researchers have introduced Expert-Sample, a novel training-free method designed to enhance the performance of fine-grained Mixture-of-Experts (MoE) models. This technique addresses the trade-off between diversity and stability in test-time scaling by analyzing the routing scores of MoE layers. Expert-Sample leverages the observation that MoE routers exhibit a high-confidence 'certain head' and a low-confidence 'uncertain tail', selectively injecting stochasticity into the latter to improve generation diversity without compromising output stability. The method has demonstrated consistent improvements in accuracy and pass@n metrics across various reasoning and coding tasks when evaluated on models like Qwen3-30B-A3B-Instruct. AI
IMPACT Introduces a training-free method to improve MoE model diversity and accuracy on reasoning and coding tasks.
RANK_REASON This is a research paper detailing a new method for improving MoE model performance.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
