A new study evaluated the real-world performance of multimodal large language models (MLLMs) in clinical dermatology, finding a significant gap between benchmark results and actual clinical utility. While models like GPT-4.1 showed promise on public datasets, their diagnostic accuracy dropped considerably when applied to a real-world cohort of 5,811 cases. Incorporating clinical context improved performance, but outputs remained sensitive to data inaccuracies, suggesting current MLLMs are not yet reliable for clinical deployment. AI
Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →
IMPACT Current multimodal LLMs show a significant performance drop in real-world clinical dermatology compared to benchmarks, indicating they are not yet ready for deployment.
RANK_REASON This is a research paper evaluating the performance of existing multimodal LLMs on a specific clinical task. [lever_c_demoted from research: ic=1 ai=1.0]