BioCreative IX MedHopQA challenges LLMs in multi-hop medical question answering

The BioCreative IX MedHopQA shared task focused on evaluating multi-hop question-answering systems in the biomedical domain. A new dataset of 1,000 QA pairs, requiring two-hop reasoning across Wikipedia pages, was created to challenge large language models, particularly for rare diseases. The competition saw 48 submissions, with the top system achieving an 89.30% F1 score on conceptual accuracy, significantly outperforming baseline models. Retrieval-augmented generation (RAG) proved crucial for high performance, and concept-level evaluation enhanced the assessment of answers. AI

IMPACT Establishes a benchmark for multi-hop medical QA, driving advancements in LLM reasoning capabilities for complex biomedical queries.

RANK_REASON The cluster describes a shared task and dataset for evaluating multi-hop question answering in the biomedical domain, which falls under research. [lever_c_demoted from research: ic=1 ai=1.0]

Read on Hugging Face Daily Papers →

• BioCreative IX
• large language models
• MedHopQA
• retrieval-augmented generation

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →