New framework optimizes VLM selection and adaptation without target labels

Researchers have developed a new framework called One Stone, Three Birds (OSTB) to address challenges in deploying vision-language models (VLMs) when target annotations are scarce. OSTB uses self-adaptive optimal transport to estimate a consensus sample-to-class structure from a pool of frozen VLMs. This learned structure then informs model selection, target adaptation, and ensembling, improving performance across various benchmarks without updating VLM parameters. AI

IMPACT Provides a novel method for VLM deployment in low-data scenarios, potentially improving efficiency and accuracy in real-world applications.

RANK_REASON Academic paper introducing a novel framework and methodology. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Optimal transport
• Vision-language models

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →