PulseAugur
中
实时 21:28:59

New benchmark SVFSearch tests multimodal LLMs on gaming video frame search

作者 PulseAugur 编辑部 · [3 个来源] ·

Researchers have introduced SVFSearch, a new benchmark designed to evaluate multimodal large language models in short-video frame search, specifically within the Chinese gaming domain. The benchmark includes 5,000 test examples and 4,198 training examples, featuring paused game scenes from real short-video clips. SVFSearch provides a controlled environment with a game-domain corpus and image gallery to ensure reproducible evaluations, revealing significant gaps between model performance and oracle knowledge, and highlighting issues in visual grounding and retrieval. AI

影响 This benchmark aims to improve multimodal LLM capabilities in understanding and retrieving information from short videos, particularly in specialized domains like gaming.

排序理由 The cluster describes a new academic paper introducing a benchmark for evaluating AI models.

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →

New benchmark SVFSearch tests multimodal LLMs on gaming video frame search

报道来源 [3]

  1. arXiv cs.AI TIER_1 English(EN) · Lingtao Mao, Huangyu Dai, Xinyu Sun, Zihan Liang, Ben Chen, Chenyi Lei, Wenwu Ou ·

    SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

    arXiv:2605.17946v2 Announce Type: replace Abstract: Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external tools, and reason over retrieved information. Yet existing benchmarks rarely ev…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

    Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external tools, and reason over retrieved information. Yet existing benchmarks rarely evaluate this ability in short-video applications, whe…

  3. arXiv cs.CV TIER_1 English(EN) · Wenwu Ou ·

    SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

    Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external tools, and reason over retrieved information. Yet existing benchmarks rarely evaluate this ability in short-video applications, whe…

相关实体

相关话题