New FedSEPT method enhances privacy in federated prompt tuning for VLMs

Researchers have introduced FedSEPT, a novel approach to federated prompt tuning for vision-language models that enhances privacy and addresses data heterogeneity. This method utilizes Subspace-decomposed Expert Modeling to create multiple prompt experts with shared low-rank factors and private residuals, limiting communication and privacy noise to a compact factor space. Additionally, FedSEPT incorporates Instance-aware Expert Fusion for adaptive expert combination and efficient fusion using cached features. Experiments across 11 diverse benchmarks demonstrate that FedSEPT achieves a superior balance between local adaptation and global generalization compared to existing methods under identical privacy constraints. AI

IMPACT Improves privacy and generalization for collaborative model adaptation in heterogeneous environments.

RANK_REASON The cluster contains a research paper detailing a new method for federated prompt tuning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Federated prompt tuning
• FedSEPT
• Hugging Face
• Instance-aware Expert Fusion
• Local differential privacy
• Multi-expert prompts
• Subspace-decomposed Expert Modeling
• Vision--language Models

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →