Study reveals document retrieval systems struggle with multi-part evidence coverage

A new study published on arXiv investigates the limitations of current document retrieval systems, particularly when dealing with multi-part requests that require evidence from different pages within a single document. The research introduces "n-Clue" as a measurement instrument to assess this gap, finding that while systems can locate relevant information, they often fail to identify all necessary pieces of evidence for a complete answer. The study highlights that even with larger models and hybrid approaches, a significant disparity persists between finding any relevant gold standard information and successfully fulfilling all conditions of a conjunctive query, indicating that condition coverage, not just gold discovery, is the primary bottleneck. AI

IMPACT Highlights a critical bottleneck in current information retrieval systems, impacting applications that require comprehensive evidence synthesis.

RANK_REASON Academic paper on information retrieval systems. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.IR (Information Retrieval) →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• Gold-NDCG
• Gotit.pub
• Hugging Face
• Influence Flower
• n-Clue
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →