New framework enhances federated cross-modal retrieval with missing modalities

Researchers have developed RCSR, a new framework designed to improve federated cross-modal retrieval, particularly when dealing with data heterogeneity and missing modalities across clients. The system utilizes a frozen CLIP backbone, incorporating shared adapters for global knowledge transfer and optional client-specific adapters for personalization. RCSR employs prototype anchoring to help unimodal clients align with global semantics and a semantic router on the server to dynamically adjust aggregation weights, enhancing both overall retrieval accuracy and training stability. AI

影响 Improves cross-modal retrieval accuracy and stability in federated learning scenarios with heterogeneous and incomplete data.

排序理由 This is a research paper detailing a new framework for federated cross-modal retrieval.

在 arXiv cs.CV 阅读 →

• arXiv
• Flickr30K
• MS-COCO

• 论文
• 其他

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →