New CSVG Task Addresses Ambiguity in Visual Dialogue Grounding

Researchers have introduced Candidate Set Visual Grounding (CSVG), a new task designed to identify the set of plausible objects referred to in ambiguous dialogue, rather than a single unique object. This approach is evaluated using the CandVG benchmark, which tracks how this candidate set evolves across dialogue turns. Initial evaluations of large-scale vision-language models reveal that while these models possess strong visual grounding capabilities, they struggle with CSVG, particularly in generating complete candidate sets and avoiding premature commitments to limited options early in conversations. However, the performance of these models can be partially improved through specialized instructions tailored for CSVG. AI

IMPACT Introduces a new benchmark for evaluating vision-language models' ability to handle ambiguous references in dialogue.

RANK_REASON The item describes a new research paper introducing a novel task and benchmark for dialogue ambiguity. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• Candidate Set Visual Grounding
• CandVG
• CatalyzeX
• CORE Recommender
• CSVG
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →