Researchers have introduced Loopie, a novel looped Transformer architecture designed to overcome previous limitations in scaling. The Loopie series includes two Mixture-of-Experts (MoE) models: a 20B-parameter version with 2B active parameters and a 6B-parameter version with 0.6B active parameters. Ablation studies indicate that Loopie significantly outperforms traditional Transformer baselines when trained with equivalent compute budgets. Furthermore, Loopie demonstrates strong reasoning capabilities, achieving gold-medal performance on the 2025 International Mathematical Olympiad and International Physics Olympiad without external tools. AI
IMPACT Sets a new benchmark for reasoning capabilities in looped Transformers, potentially influencing future model development.
RANK_REASON The cluster describes a new research paper detailing a novel AI model architecture and its performance on academic competitions.
Read on Hugging Face Daily Papers →
- 30B-A3B
- International Mathematical Olympiad
- International Physics Olympiad
- Loopie
- mixture of experts
- Transformer++
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
