R&Q

New R&Q strategy balances SMoE LLM load without retraining

Researchers have developed a new inference-time strategy called Replicate-and-Quantize (R&Q) to address load imbalance in Sparse Mixture-of-Experts (SMoE) large language models. This method dynamically rebalances worklo…