New CV-OPD method enhances VLM visual reasoning by correcting shortcuts

Researchers have developed a new method called Contrastive Vision On-policy Distillation (CV-OPD) to improve the visual reasoning capabilities of Vision-Language Models (VLMs). Existing on-policy distillation techniques were found to allow VLMs to generate correct answers without genuinely relying on visual evidence, a shortcut attributed to a lack of visual constraints. CV-OPD addresses this by contrasting teacher distributions from positive and negative examples, using a discrepancy-aware gate to focus on visually relevant information and a contrastive divergence loss to correct the model's exploitation of visual regions. Experiments show CV-OPD significantly enhances the visual reasoning of VLMs across various benchmarks. AI

IMPACT Enhances VLM visual reasoning by addressing shortcut learning, potentially leading to more reliable AI systems in visual tasks.

RANK_REASON The cluster contains a research paper detailing a new method for improving Vision-Language Models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Contrastive Divergence Loss
• Contrastive Vision On-policy Distillation
• CV-OPD
• Discrepancy Aware Gate
• On-Policy Distillation
• Vision-Language Models

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →