IFM has released K2-Horizon-MoVA-36B-A4B, a Mixture-of-Experts model with Mixture-of-Values attention. This model boasts 36 billion parameters but actively uses only 4 billion per token, achieving frontier-class results on agentic and reasoning benchmarks. It significantly outperforms open-weight dense and MoE models up to 15 times its size and is competitive with closed frontier models. The model supports a 512K context window, and IFM plans to release intermediate checkpoints, training data, and code for full transparency. AI
IMPACT Sets a new benchmark for MoE models in terms of parameter efficiency and context length, potentially influencing future model architectures.
RANK_REASON Frontier-lab model release with system card and benchmark results. [lever_c_demoted from frontier_release: ic=2 ai=1.0]
Read on Hugging Face Trending Models →
- Docker
- Google Colab
- Hugging Face
- IFM/K2-Horizon-MoVA-36B-A4B
- K2-Horizon-MoVA-36B-A4B
- Kaggle
- MOVA
- OpenAI
- SGLang
- transformers
- vLLM
AI-generated summary · Google Gemini · from 3 sources. How we write summaries →