Researchers have introduced DiagTrace, a new benchmark designed to evaluate large language models (LLMs) on their ability to perform interactive clinical decision-making. Unlike previous benchmarks that provide all patient information at once, DiagTrace simulates real-world diagnosis by requiring LLMs to incrementally seek information. The benchmark is built from 510 case reports sourced from PubMed Central, covering eight medical specialties. Initial evaluations of four LLMs using DiagTrace revealed trade-offs in their ability to align with clinical courses, manage interactions, and maintain efficiency. AI
IMPACT This benchmark could drive the development of LLMs better suited for interactive, evidence-gathering diagnostic tasks in healthcare.
RANK_REASON The cluster describes a new benchmark for evaluating LLMs in a specific domain, presented in an academic paper. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →