OVHcloud has released Olala-v1.0-mini-7A1B, a 6.8 billion parameter mixture-of-experts model with approximately 1 billion active parameters per token. The model is available on Hugging Face under the Apache 2.0 license, with various checkpoints including base, SFT, and post-trained versions. Initial reports suggested an 8 GB memory requirement, but the BF16 file alone is 13.59 GB, with quantized versions like Q5_K_M reducing the size to around 5.99 GB. The model's performance claims imply a need for significant memory bandwidth, and self-hosters will need specific forks of popular LLM loading software as current versions do not yet support its architecture. AI
IMPACT Requires specific software forks for self-hosting, indicating potential compatibility challenges for users of standard LLM tools.
RANK_REASON The item details the release of a new LLM, Olala-v1.0-mini-7A1B, including its technical specifications, memory requirements, and performance claims, which falls under research and model release. [lever_c_demoted from research: ic=1 ai=1.0]
- Apache Software License 2.0
- GGUF
- Hugging Face
- Jean Zay
- Jupiter
- L4
- L40S
- llama.cpp
- LM Studio
- MeluXina - Accelerator Module
- Mlx
- NVIDIA GH200
- NVIDIA H100
- Olala-v1.0-mini-7A1B
- Ollama
- OVHaiLLM
- OVHcloud
- vLLM
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →