ExactMoE

ExactMoE slashes MoE model memory use by 87% with minimal accuracy loss

Researchers have developed ExactMoE, a novel inference design for sparse mixture-of-experts (MoE) language models that significantly reduces memory requirements. By applying four-bit weight quantization only to the acti…