A new benchmark called FOCUS has been developed to evaluate the generalization capabilities of AI models in retinal image analysis. This benchmark assesses vision-only encoders, vision-language dual-encoder models, and multimodal large language models across ten public datasets. The evaluation measures ranking performance, calibration, subgroup disparities, and robustness to image quality variations. Results indicate that while general vision encoders perform well in ranking, multimodal models show variability and benefit from fine-tuning, though this does not always translate to improved calibration on external datasets. AI
IMPACT Establishes a new standard for evaluating AI model reliability and fairness in medical imaging, potentially guiding future development and deployment.
RANK_REASON The cluster contains a research paper detailing a new benchmark for AI models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
