PulseAugur
EN
LIVE 20:32:06

ModaLens audit reveals medical VLMs rely more on text than images

By PulseAugur Editorial · [2 sources] ·

Researchers have developed ModaLens, a new audit method to measure the sensitivity of vision-language models (VLMs) in medical contexts. Using the MedGemma-27B model on MIMIC-CXR data, the study found that when radiology reports are available, the model's reliance on image data decreases significantly. Specifically, the model's answers changed only 4.26% of the time when reports were present, compared to 20.94% when reports were absent, indicating that the textual report heavily influences the model's output, potentially at the expense of visual interpretation. AI

IMPACT Highlights potential over-reliance on text in medical VLMs, suggesting a need for improved image-grounding in clinical applications.

RANK_REASON The cluster contains an academic paper detailing a new methodology and experimental results for evaluating AI models.

Read on Hugging Face Daily Papers →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

ModaLens audit reveals medical VLMs rely more on text than images

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
The cluster contains an academic paper detailing a new methodology and experimental results for evaluating AI models.
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
12 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

Full methodology in our editorial standards.

COVERAGE [2]

  1. arXiv cs.AI TIER_1 English(EN) · Sebasti\'an Andr\'es Cajas Ord\'o\~nez, Maximin Lange, Quang Bui, Anqi Peter Li, Felipe Ocampo Osorio, Rafi Al Attrach, Kushul Reddy Palakala, Sahil Kapadia, Zakaria Laouabdia Sellami, Xinyue Zhang, Ashley Zhang, Leo Anthony Celi ·

    ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

    arXiv:2609.15635v1 Announce Type: cross Abstract: A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a paired image-swap audit, measures how report availability changes image sensitivity:…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs

    A paired image-swap audit reveals that report availability substantially reduces vision-language model sensitivity to image changes in radiology question answering.

RELATED ENTITIES

RELATED TOPICS