TallyTrain protocol slashes federated learning communication costs

Researchers have developed TallyTrain, a novel federated learning protocol designed to significantly reduce communication overhead. This method transmits only the predicted class index for each probe, rather than full soft labels, which is particularly effective for large class counts. TallyTrain can outperform traditional soft-label distillation in non-independent and identically distributed (non-IID) scenarios by filtering out noise from under-trained peers. Additionally, a bandwidth-bridging variant combines TallyTrain with sparse parameter merges, outperforming standard baselines like FedAvg and FedProx across various operating points. AI

IMPACT Reduces communication overhead in federated learning, potentially enabling larger models and class counts in distributed training scenarios.

RANK_REASON Academic paper detailing a new method for federated learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• FedAvg
• FedDF
• Federated learning
• FedProx
• Radhakrishna Achanta
• TallyTrain

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →