Spatiotemporal Router

New framework boosts MoE model inference efficiency

Researchers have developed a cache-aware framework to improve the memory efficiency of Mixture-of-Experts (MoE) models during inference. The proposed post-training method jointly adapts the MoE backbone and lightweight …