The Allen Institute for Artificial Intelligence (AI2) has released Olmo-core 3, an open-source training infrastructure designed for large mixture-of-experts (MoE) models. This system is foundational for the development of future Olmo models, with capabilities to scale up to trillion-parameter sizes. The release includes a GitHub repository for the infrastructure. AI
IMPACT Provides open-source infrastructure for training large MoE models, potentially accelerating research and development in this area.
RANK_REASON Release of open-source training infrastructure for large models by a research institute. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Bluesky Jetstream — AI desk →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →