Two new research papers propose novel methods for federated parameter-efficient fine-tuning (PEFT) to address communication bottlenecks and improve model performance on decentralized data. The first paper introduces TRISHUL, a spectral-control framework that uses shared frozen bases and nuclear norm shrinkage to aggregate client updates robustly, showing gains on vision and language benchmarks with LLaMA3.2-1B. The second paper presents FLITE, which leverages mapping networks with a low-rank factorization and delta formulation to achieve significant reductions in communication payload, demonstrating strong performance on CIFAR-100 with ResNet-18, even with extreme data compression. AI
IMPACT These methods aim to significantly reduce communication overhead in federated learning, potentially enabling more efficient training of large models on decentralized devices.
RANK_REASON Two academic papers published on arXiv detailing novel methods for federated learning.
Read on Hugging Face Daily Papers →
- arXiv
- CIFAR-100
- FedAvg
- FLITE
- GroupNorm
- Radhakrishna Achanta
- ResNet-18
- Dipsan Bhattarai
- Federated parameter-efficient fine-tuning
- LLaMA3.2 1B
- TRISHUL
AI-generated summary · Google Gemini · from 3 sources. How we write summaries →
