Researchers have introduced Decentralized Primal Averaging (DPA), a new algorithm designed for training deep learning models across networks where data is kept local to each agent. DPA addresses the challenge of data heterogeneity, which can significantly impact model performance. The algorithm employs a local quasi-global momentum buffer to improve optimization without requiring agents to transmit auxiliary tracking variables. Experiments on image classification benchmarks demonstrate that DPA outperforms existing methods, particularly under conditions of high data heterogeneity and sparse communication, achieving higher test accuracy with comparable communication costs. AI
IMPACT Enhances the feasibility of training large models on distributed, heterogeneous datasets without centralizing data.
RANK_REASON Academic paper detailing a new algorithm for decentralized deep learning. [lever_c_demoted from research: ic=1 ai=1.0]
- CIFAR-10
- CIFAR-100
- Decentralized Primal Averaging
- Diyako Ghaderyan
- DSGD
- QG-DSGDm
- Quasi-Global Momentum
- ResNet-20
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
