SPARK method accelerates decentralized federated learning with stable NTK updates

Researchers have developed SPARK, a novel method to improve the convergence speed and stability of decentralized federated learning (DFL) under heterogeneous data conditions. SPARK utilizes a stage-wise annealed soft-label regularizer combined with momentum to accelerate neural tangent kernel (NTK) updates, which traditionally struggle with instability in such scenarios. The proposed approach demonstrates significant improvements, achieving up to a 3x faster convergence rate and reducing communication by approximately 70% compared to existing baselines, while also maintaining higher accuracy across various data distributions and network setups. AI

IMPACT Enhances efficiency and stability in decentralized AI model training, potentially enabling more robust collaborative learning across diverse datasets.

RANK_REASON The cluster contains a research paper published on arXiv detailing a new method for federated learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• SPARK
• arXiv
• DagsHub
• Decentralized federated learning
• federated learning
• Hugging Face
• IArxiv
• Li Xia
• neural tangent kernels

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →