Wang Wenfeng

Specter system accelerates MoE inference with quantization and speculative offloading

Researchers have developed Specter, a novel inference system designed to accelerate Mixture of Experts (MoE) models, which often exceed the memory capacity of a single accelerator. Specter combines quantization with spe…