IFM has released K2-Horizon, a suite of six open-weight language models ranging from 0.9B to 375B parameters. This release is notable for its comprehensive openness, including not only the final model weights but also intermediate checkpoints, training data, logs, and infrastructure code. The models are designed for general reasoning, coding, and agentic tasks, with the flagship 375B-A23B MoE model featuring 23B active parameters. Key technical innovations include MoVA (Mixture-of-Value Attention) and Softplus attention gating, which reportedly allow smaller MoE models to achieve performance comparable to larger dense models with fewer active parameters. AI
IMPACT This comprehensive open release could set a new standard for transparency and reproducibility in LLM development, potentially accelerating research and independent evaluation.
RANK_REASON Frontier-lab model release with full training data and code. [lever_c_demoted from frontier_release: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →