Moonshot AI has released the Kimi K3 model with 2.8 trillion open weights, making it the largest open-weight model to date. However, only a small fraction, approximately 1.8%, of these parameters are actively used for processing each token. This sparsity means that while the model boasts a massive parameter count, the actual computational and memory requirements for inference are significantly lower than a dense model of equivalent size, though still substantial, requiring around 1488 GB of VRAM. AI
IMPACT This release highlights a trend towards sparse models, potentially reducing inference costs and hardware requirements for large-scale AI.
RANK_REASON Frontier-lab model release with system card. [lever_c_demoted from frontier_release: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
