New benchmark reveals MLLMs struggle with spatial reasoning

Researchers have introduced PCSR-Bench, a new diagnostic benchmark designed to evaluate the spatial reasoning capabilities of multimodal large language models (MLLMs) when processing omnidirectional images. The benchmark, comprising over 84,000 question-answer pairs across 2,600 images, reveals a significant gap between foundational perception and advanced reasoning tasks. While models perform moderately well on basic tasks like object counting, their accuracy plummets on more complex reasoning involving viewpoint changes and egocentric distortions. Further experiments using reinforcement learning on a smaller model indicate that spatial reasoning abilities can be improved through targeted optimization, though gains are task-specific and sensitive to reward design. AI

影响 Highlights a key bottleneck in current MLLMs, suggesting a need for improved spatial reasoning capabilities for more robust AI applications.

排序理由 The cluster describes a new academic paper introducing a benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CV 阅读 →

• 360-degree omnidirectional images
• Multimodal Large Language Models
• PCSR-Bench

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →