A smaller, 0.4 billion parameter version of the moonshotai/Kimi-K3 model has been released on Hugging Face for testing and development purposes. This tiny model, named inference-optimization/Kimi-K3-0.40B, retains key architectural features of its larger counterpart, including a 3:1 ratio of KimiDeltaAttention to KimiMLAAttention layers and attention residual connections. It was created using the LLM-Compressor tool and fine-tuned on a small dataset, resulting in a significantly reduced parameter count while maintaining some generative capabilities. AI
IMPACT Provides a smaller, more accessible version of Kimi-K3 for developers to experiment with and integrate into applications.
RANK_REASON Release of a smaller, experimental version of an existing model for testing and development, not a full frontier release.
Read on Hugging Face Trending Models →
- inference-optimization/Kimi-K3-0.40B
- KimiDeltaAttention
- KimiK3ForConditionalGeneration
- KimiMLAAttention
- LLM-Compressor
- moonshotai/Kimi-K3
- transformers
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
