ReportQA: QA-Based Radiology Report Evaluation
Researchers have introduced ReportQA, a novel framework for evaluating radiology report generation systems. This framework leverages large language models (LLMs) to extract structured information from reports and generate question-answer pairs. The QAScore metric, derived from an LLM's accuracy in answering these questions, demonstrates better alignment with radiologist judgments than existing metrics. Experiments using this framework revealed that current vision-language models struggle with fine-grained clinical representations, suggesting that question-driven inference is a more effective approach for report generation. AI