Zyphra has released ZAYA1-8B, an 8.4 billion parameter Mixture-of-Experts model that only activates approximately 760 million parameters per token. This architecture allows it to achieve performance comparable to much larger models on math and coding benchmarks, including Claude 4.5 Sonnet. The model incorporates architectural changes like Compressed Convolutional Attention and an MLP-based router for expert selection, and was trained on a large cluster of AMD Instinct MI300x nodes. AI
IMPACT Achieves frontier-level performance with significantly reduced active parameters, potentially lowering inference costs for advanced models.
RANK_REASON Model release from a lab with novel architectural innovations. [lever_c_demoted from frontier_release: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
