Asynchronous Execution Engine

MoE-SpAc framework enhances MoE model inference efficiency on edge devices

Researchers have developed MoE-SpAc, a novel framework designed to improve the efficiency of Mixture-of-Experts (MoE) model inference on heterogeneous edge devices. This system repurposes speculative decoding techniques…