Researchers have introduced OmniMapBench, a new benchmark designed to evaluate the visual-centric reasoning capabilities of Large Vision-Language Models (LVLMs). This benchmark addresses a limitation in existing datasets where visual information can often be reduced to text, thus not truly testing visual grounding. OmniMapBench features 2,096 question-answer pairs across 1,603 map documents and introduces the Visual Dependency Index (VDI) to quantify the irreducibility of visual reasoning. Initial evaluations on 25 leading LVLMs revealed a significant performance gap, with the top model achieving only 75.03% accuracy, highlighting the challenges for current LVLMs in this domain. AI
IMPACT This benchmark aims to drive progress in LVLM visual reasoning, potentially leading to more capable AI systems for interpreting complex visual documents.
RANK_REASON The cluster describes a new academic benchmark and research paper published on arXiv.
- alphaXiv
- arXiv
- DagsHub
- Hugging Face
- LVLMs
- OmniMapBench
- Visual Dependency Index
- CatalyzeX
- Gotit.pub
- ScienceCast
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
