New Benchmark and Agentic Framework Enhance Audio-Visual Reasoning in LLMs

Researchers have introduced MOV-Bench, a new benchmark designed to evaluate multi-hop audio-visual reasoning capabilities in Omni-LLMs. This benchmark features 519 questions requiring complex reasoning over dispersed audio and visual evidence. To address the limitations of current models, the team also developed AOP-Agent, an agentic framework that enhances Omni-LLMs' active perception abilities without requiring additional training. Experiments show AOP-Agent significantly improves reasoning performance, especially on longer videos and more demanding questions. AI

IMPACT Introduces a new benchmark and framework to push the boundaries of multi-hop audio-visual reasoning in LLMs.

RANK_REASON This is a research paper introducing a new benchmark and an agentic framework for AI reasoning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• AOP-Agent
• MOV-Bench
• Omni-LLMs
• OmniVideoBench

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →