A 34.7 billion parameter Mixture-of-Experts (MoE) model, Ourbox-35B-JGOS, has demonstrated the ability to run efficiently on a CPU, a feat typically associated with much smaller models. This efficiency is attributed to its sparse architecture, where only a fraction of the model's parameters are active per token, significantly reducing memory bandwidth requirements compared to dense models. The model achieves impressive performance on benchmarks like GPQA Diamond, and its weights and performance metrics are publicly available for reproduction. AI
IMPACT Demonstrates that large MoE models can be made accessible on low-resource hardware, potentially broadening the reach of advanced AI capabilities.
RANK_REASON The item details a technical approach to running a large MoE model on consumer hardware, including benchmark results and reproducible methods. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
