A new research paper from arXiv explores the concept of grounded reasoning in video-language models. The study introduces a method called the 'cite gap' to test whether a model's answer actually depends on the specific frames and regions it cites. By intervening and blacking out these cited areas, researchers found that for models trained on CLEVRER, correct answers flipped significantly more often than in control experiments, indicating a dependency on cited regions. However, models trained on NExT-QA showed a much smaller cite gap, suggesting that while citations might be verifiable, the answers may not always depend on them. AI
IMPACT Introduces a novel method to evaluate the true dependency of model outputs on cited evidence, potentially improving the reliability of video-language models.
RANK_REASON Academic paper published on arXiv detailing a new evaluation method for video-language models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →