Moonshot AI launches 2.8T parameter Kimi K3 model with 1M context window

Moonshot AI has launched Kimi K3, a 2.8 trillion parameter model featuring Kimi Delta Attention and Attention Residuals. This model natively supports visual understanding and boasts a 1 million token context window, making it the first open-source model of its scale. Kimi K3 is designed for advanced applications such as long-form programming, knowledge work, and complex reasoning. AI

IMPACT Sets new SOTA for open-source models in context window and parameter count, potentially accelerating research in long-context applications.

RANK_REASON Frontier-lab model release with system card. [lever_c_demoted from frontier_release: ic=1 ai=1.0]

Read on 36氪 (36Kr) →

• Attention Residuals
• Kimi Delta Attention
• Kimi K3
• Moonshot AI

• model release
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →