A new study published on arXiv evaluated the diagnostic accuracy and navigation behavior of vision-language model (VLM) agents when interpreting emergency CT and MRI scans. Across various exploration strategies, the VLMs demonstrated significantly lower accuracy compared to human radiologists, with pooled accuracy ranging from 32.6% to 42.5% depending on the navigation method. The models also showed a tendency to form an early hypothesis and rarely correct it, despite expressing high confidence in their final answers. AI
IMPACT Vision-language models show limited diagnostic capabilities in medical imaging, highlighting the need for further development and human oversight.
RANK_REASON The cluster contains a research paper detailing the performance of AI models on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- Bastien Le Guellec
- computed tomography
- DagsHub
- Gotit.pub
- Hugging Face
- magnetic resonance imaging
- ScienceCast
- vision-language model
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →