Luce Spark is a new open-source system that enables large 35 billion parameter Mixture-of-Experts (MoE) models to run on a single 16 GB GPU. It achieves this by intelligently keeping only the currently active experts on the GPU, while the rest reside in system RAM and are swapped in as needed. This approach avoids the performance penalty typically associated with offloading, allowing models that would otherwise not fit to run efficiently. AI
IMPACT Enables running large MoE models on consumer hardware, democratizing access to advanced AI capabilities.
RANK_REASON The cluster describes a new open-source system and methodology for running large MoE models on limited hardware, which is a significant research contribution in the field of efficient AI deployment.
- 16 GB GPU
- Laguna XS.2 33B-A3B
- Luce-Org/lucebox-hub
- Luce Spark
- Qwen3.6 35B-A3B
- RTX 3090
- Apache2.0
- GPU
- llama.cpp
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
