A new study published on arXiv reveals that current Video Large Language Models (Video-LLMs) struggle with accurately tracking characters throughout long videos. Despite strong performance on benchmarks like InfiniBench's global appearance task, the models often fail to identify specific characters, especially when distinguishing between individuals of the same gender. The research suggests that these models rely on superficial cues like gender rather than deep character recognition, leading to significant drops in accuracy when questions are posed open-endedly or when character names are altered. AI
IMPACT Reveals limitations in current Video-LLMs, suggesting a need for improved character-tracking mechanisms beyond superficial cues.
RANK_REASON The cluster contains an academic paper detailing research findings on Video-LLM capabilities.
AI-generated summary · Google Gemini · from 3 sources. How we write summaries →
