16 GB GPU

Luce Spark enables 35B MoE models on 16GB GPUs

Luce Spark is a new open-source system that allows large Mixture-of-Experts (MoE) language models, specifically 33-35 billion parameters, to run on a single 16GB GPU. It achieves this by intelligently keeping only the c…