A new study published on arXiv evaluates the performance of multimodal decision models, specifically CLEF and CLEF-Flash, in classifying breast ultrasound images. The research utilized three datasets (Dataset B, BUSI, and BUS-BRA) and tested various prompts to assess accuracy, F1 scores, and latency. CLEF-Flash with a clinical prompt showed the highest accuracy across datasets, though CLEF with a task-guided prompt performed better on specific metrics for the BUSI dataset. The study highlighted significant dataset and prompt dependencies, noting frequent benign-to-malignant misclassifications, particularly on the BUS-BRA dataset, where performance often fell below that of a simple majority-class predictor. AI
IMPACT Highlights the need for careful evaluation of multimodal models in medical imaging, as performance is highly dependent on the dataset and prompt used.
RANK_REASON Academic paper evaluating multimodal models on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →