VLMs fail to re-examine images when prompted, study finds

Researchers have developed a new framework called VisualSwap to test whether Vision-Language Models (VLMs) truly re-examine images when they claim to. Their experiments using the VS-Bench dataset on models like Qwen3-VL and Kimi-VL showed that these models frequently fail to detect semantic changes in images, even when visually similar. This suggests that VLMs often generate text about visual re-examination without actually performing it, a tendency exacerbated in models designed for more complex reasoning. AI

影响 Challenges the perceived visual understanding of current VLMs, suggesting a need for improved grounding mechanisms beyond textual cues.

排序理由 Academic paper introducing a new framework and dataset for evaluating VLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CV 阅读 →

• ERNIE-VL
• Kimi-VL
• Qwen3-VL
• VisualSwap
• VS-Bench

• 论文
• 安全

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →