Researchers have introduced ListQA, a new benchmark designed to evaluate the ability of large language models (LLMs) to retrieve and organize factual information into lists. The benchmark comprises over 9,000 human-curated questions that require models to recall multiple facts, with an audited error rate of 1.7%. Initial testing on 34 models revealed that even the best-performing models struggle with multi-faceted questions and retrieving facts solely from their parameters, though providing source documents like Wikipedia pages significantly improves performance. AI
IMPACT This benchmark could drive improvements in LLM's ability to handle complex, multi-part factual queries, enhancing their utility in knowledge-intensive applications.
RANK_REASON The cluster describes a new academic benchmark for evaluating LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- Hugging Face
- Large Language Models
- ListQA
- National Collegiate Athletic Association
- Rakesh Vaideeswaran Mahesh
- Wikipedia
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
