Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning
Researchers have developed SPARK, a novel method to improve the convergence speed and stability of decentralized federated learning (DFL) under heterogeneous data conditions. SPARK utilizes a stage-wise annealed soft-label regularizer combined with momentum to accelerate neural tangent kernel (NTK) updates, which traditionally struggle with instability in such scenarios. The proposed approach demonstrates significant improvements, achieving up to a 3x faster convergence rate and reducing communication by approximately 70% compared to existing baselines, while also maintaining higher accuracy across various data distributions and network setups. AI
IMPACT Enhances efficiency and stability in decentralized AI model training, potentially enabling more robust collaborative learning across diverse datasets.