A new benchmark called OccludeBench has been developed to test the ability of vision-language models (VLMs) to infer missing or occluded objects in images. The benchmark revealed that current VLMs perform significantly worse at recovering directly occluded objects (42.7% accuracy) compared to inferring from broader scene context (62.7% accuracy), while humans show minimal difference between these tasks. A diagnostic tool, OccludeTrace, identified that VLMs struggle most when recovering an object from a specific visual cue like a shadow or reflection. Prompting techniques and tool use showed inconsistent improvements, but an intervention involving inpainting a candidate object before evaluation significantly reduced the performance gap for models with image-generation capabilities. AI
IMPACT Highlights a specific weakness in VLM visual reasoning, potentially guiding future model development for better contextual understanding and object inference.
RANK_REASON The cluster contains a new academic paper introducing a benchmark and diagnostic tool for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
