New MIRA-Ev benchmark evaluates clinical NLP models on evidence reasoning

Researchers have introduced MIRA-Ev, a new benchmark designed to evaluate clinical Natural Language Processing (NLP) models. Unlike existing methods that focus solely on final answer accuracy, MIRA-Ev assesses how models utilize evidence from clinical texts. The benchmark is built upon Spanish Médico Interno Residente (MIR) licensing exam cases and includes detailed annotations for premises, claims, and their supporting or attacking relations. MIRA-Ev is available in Spanish, English, and Basque, marking the first clinical argumentation resource in Basque. AI

IMPACT This benchmark could lead to more robust clinical AI systems that better understand and utilize evidence in patient cases.

RANK_REASON The cluster describes a new academic benchmark for evaluating AI models, published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Basque
• Hugging Face

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →