OVHcloud has released Olala mini 7A1B, an open-source large language model designed for efficient operation. This model features 7 billion parameters with 1 billion activated, utilizing a Mixture-of-Experts architecture with 256 experts. It can achieve speeds of 160 tokens per second on a GPU or 50 tokens per second on a CPU, and requires as little as 8GB of VRAM, making it accessible for consumer-grade hardware. The model is available under the Apache 2.0 license. AI
IMPACT Enables efficient LLM deployment on consumer hardware, potentially broadening access to AI capabilities.
RANK_REASON Frontier-lab model release with system card [lever_c_demoted from frontier_release: ic=1 ai=1.0]
Read on Mastodon — fosstodon.org →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →