New framework enhances zero-shot sketch-based image retrieval

Researchers have developed SeCo-SBIR, a new framework for zero-shot sketch-based image retrieval (ZS-SBIR) that aims to improve generalization by bridging the domain gap between sketches and photos. The framework uses a text-guided multi-modal prompting strategy to inject semantic knowledge from CLIP's text encoder into its visual encoder. Additionally, a perturbation-based consistency constraint aligns the adapted model with a frozen CLIP reference, preventing overfitting and anchoring learned representations. This approach achieves state-of-the-art results on standard ZS-SBIR benchmarks. AI

IMPACT This research could improve the accuracy and generalization of AI systems that rely on visual understanding and retrieval from sketches.

RANK_REASON The cluster contains an academic paper detailing a new method for image retrieval. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• Hugging Face
• InfoNCE
• NT-Xent
• SeCo-SBIR
• ZS-SBIR

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →