Zyphra has released ZAYA1-8B, a reasoning-focused mixture-of-experts model with 700 million active parameters. The model was trained from scratch on an AMD compute platform and utilizes a novel four-stage reinforcement learning cascade. ZAYA1-8B demonstrates competitive performance on mathematics and coding benchmarks, even when compared to significantly larger models, by employing a reasoning-focused training methodology and an answer-preserving trimming scheme. AI
IMPACT This model's performance on reasoning benchmarks, particularly with a limited active parameter count, suggests potential for more efficient reasoning models.
RANK_REASON This is a technical report detailing a new model release from a non-frontier lab. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
