New HE-OFT protocol enables privacy-preserving federated LLM fine-tuning

Researchers have developed HE-OFT, a novel protocol for privacy-preserving federated fine-tuning of large language models. This method allows multiple parties to collaboratively fine-tune a model without any participant receiving the fully trained model, addressing privacy concerns related to regulated or proprietary assets. HE-OFT achieves this by having clients fine-tune only a low-rank adapter and classifier head, uploading encrypted contributions that are combined using multiparty CKKS encryption without decryption by the server. The protocol demonstrates strong performance, maintaining 85-96% of the accuracy of disclosed models while significantly reducing data transfer requirements. AI

IMPACT This research could enable more secure collaborative AI development, particularly in regulated industries where model privacy is paramount.

RANK_REASON The cluster describes a new research paper detailing a novel protocol for federated learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• CKKS
• federated learning
• Halil İbrahim Kanpak
• HE-OFT
• homomorphic encryption

• paper
• safety
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →