DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections
Researchers have introduced DocHop-QA, a new benchmark designed to evaluate multi-hop reasoning capabilities over multimodal scientific documents. This benchmark addresses the limitations of existing QA datasets by incorporating text, tables, and layout cues from multiple PubMed articles, simulating real-world scientific information seeking. Current large language models demonstrate significant challenges in handling the long-context and multi-evidence requirements of DocHop-QA, highlighting its potential as a rigorous testbed for future advancements in scientific QA systems. AI
IMPACT Establishes a new benchmark for evaluating multimodal, multi-document reasoning in LLMs, pushing the frontier for scientific information retrieval.