This paper introduces a novel decentralized federated distillation method designed for clients with heterogeneous models. The approach leverages shared unlabeled public data for collaboration, where each client evaluates predictions across three modalities: class prediction, boundary decision, and prediction correlation. Unreliable clients are filtered, and weighted based on their reliability, to construct specialized teachers for each knowledge type. The method validates distillation gradients against supervised gradients from private data, removing conflicting information and suppressing relation gradients before updating the model. Experiments on CIFAR-10 and CIFAR-100 datasets show improved prediction accuracy for heterogeneous models under non-IID data and Byzantine attacks, indicating potential for adoption in unreliable real-world decentralized environments. AI
IMPACT This research could improve the robustness and accuracy of federated learning systems in decentralized environments with heterogeneous models and potential malicious actors.
RANK_REASON The cluster contains an academic paper detailing a new research method. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
