Researchers have developed a new method called OptEP for pruning Mixture-of-Experts (MoE) language models, which significantly reduces the number of expert parameters that need to be stored without sacrificing performance. This technique treats expert selection as a subset-selection problem and uses a scalable continuous relaxation approach to optimize expert choices and update retained weights. Experiments show OptEP achieves substantial accuracy gains, up to 15.05% on Qwen3-30B-A3B, especially under aggressive pruning scenarios, demonstrating its effectiveness in maintaining model performance while reducing storage requirements. Separately, a study explored the use of small recurrent-depth language models for multitask clinical text processing, finding that models like Ouro-1.4B-Thinking performed competitively with larger transformer models on various tasks, suggesting potential for simplified deployment and improved knowledge transfer in clinical NLP. AI
IMPACT New pruning techniques could enable more efficient deployment of large MoE models, while recurrent models offer potential for specialized clinical NLP tasks.
RANK_REASON Two distinct research papers published on arXiv detailing new methods and model explorations in NLP.
- arXiv
- GLM-4.5-Air
- HRM-Text-1B
- Hugging Face
- Jelena Markovic-Voronov
- Mixtral-8x7B
- Mixture-of-Experts
- OptEP
- Ouro1.4B-Thinking
- Ouro-2.6B-Thinking
- Qwen3-1.7B
- Qwen3-235B-A22B
- Qwen3-30B-A3B
- Qwen3-8B
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
