Memory-Efficient Partitioned DNN Inference on Resource-Constrained Android Crowds
Researchers have developed a new system called CROWD IO to enable the efficient inference of large deep neural networks on resource-constrained Android devices. The system addresses the challenge of limited RAM on mobile phones by distributing memory pressure across multiple devices. CROWD IO employs several mechanisms, including deferred partition loading and compressed tensor transport, to manage memory usage and reduce batch latency. AI
IMPACT Enables deployment of advanced AI models on a wider range of mobile devices, potentially increasing edge AI capabilities.