Can Agents Distinguish Visually Hard-to-Separate Diseases in a Zero-Shot Setting? A Pilot Study
A pilot study published on arXiv explores the capability of multimodal large language models (MLLMs) to distinguish between visually similar diseases in a zero-shot setting. Researchers introduced a multi-agent framework using contrastive adjudication to test agents on diagnostic tasks for melanoma versus atypical nevus and pulmonary edema versus pneumonia. While the framework showed an 11-percentage-point gain in accuracy on dermoscopy data and reduced unsupported claims, the overall performance is not yet sufficient for clinical deployment due to limitations like the absence of clinical context and inherent uncertainty in human annotations. AI
IMPACT This research highlights the potential for MLLMs in medical diagnostics, though significant improvements are needed before clinical application.