New framework enhances federated cross-modal retrieval with missing modalities

Researchers have developed RCSR, a new framework designed to improve federated cross-modal retrieval, particularly when dealing with data heterogeneity and missing modalities across clients. The system utilizes a frozen CLIP backbone, incorporating shared adapters for global knowledge transfer and optional client-specific adapters for personalization. RCSR employs prototype anchoring to help unimodal clients align with global semantics and a semantic router on the server to dynamically adjust aggregation weights, enhancing both overall retrieval accuracy and training stability. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Improves cross-modal retrieval accuracy and stability in federated learning scenarios with heterogeneous and incomplete data.

RANK_REASON This is a research paper detailing a new framework for federated cross-modal retrieval.

Read on arXiv cs.CV →

• RCSR
• CLIP
• MS-COCO
• Flickr30K
• arXiv

• paper
• other