CANS: Accelerating Multiuser Collaborative Edge Inference via Cooperative Autodidactic NeuroSurgeon
Researchers have developed a new framework called Cooperative Autodidactic NeuroSurgeon (CANS) to improve the efficiency of collaborative deep neural network inference on mobile edge devices. CANS allows devices to adaptively learn optimal model partitions by sharing feedback during inference, addressing challenges posed by fluctuating network conditions and diverse device capabilities. The framework incorporates a FedLinUCB-DW algorithm for device grouping and leverages offline experience for faster exploration, with theoretical guarantees on its performance. In prototype experiments, CANS demonstrated a significant reduction in inference latency, cutting it by up to 50% compared to non-cooperative methods. AI
IMPACT Optimizes collaborative edge inference, potentially reducing latency and improving user experience for mobile AI applications.