CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluations
Researchers have developed CardioLens, a new evaluation testbed for multimodal large language models (MLLMs) using multi-sequence cardiac MRI data. The testbed, constructed from private hospital archives, contains over 473,000 slices and 13,000 verified question-answer pairs across various MRI sequences. Evaluations using CardioLens revealed a significant gap between MLLM performance on public benchmarks and their actual clinical utility, with models struggling to integrate information across different sequences and temporal phases. AI
IMPACT Highlights the limitations of current MLLMs in complex clinical settings, indicating a need for models that can better integrate multi-modal, sequential data for real-world applications.