Vision-language models for chest radiography do not always need the image
A new study published on arXiv questions the reliance on image data for vision-language models in chest radiography. Researchers developed a causal audit to test whether these models truly utilize image information or if they rely on text-based priors. The findings indicate that some models, including a large 119-billion-parameter model, perform similarly to text-only baselines, suggesting they may ignore image data. The study proposes that grounding audits, rather than accuracy metrics alone, should be used to evaluate and approve these models for clinical use. AI
IMPACT Highlights the need for robust auditing of AI models in critical domains like healthcare, suggesting current accuracy metrics may be misleading.