English(EN) Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

新指标诊断医学VLM的模态不平衡

作者 PulseAugur 编辑部 · [2 个来源] · 2026-06-03 08:50

研究人员开发了一种名为频谱对齐分数（SAS）的新指标，用于诊断视觉语言模型（VLM）中的模态不平衡，尤其是在医学领域。与现有的对称指标不同，SAS提供方向性分数，以识别是哪种模态（图像或文本）导致性能下降。在医学和自然数据集上的15个VLM上进行的实验表明，与文本描述相比，SAS能有效捕捉医学图像中更丰富的信息，并且在与检索性能的相关性方面优于其他指标。 AI

影响为提高医学VLM的可靠性提供了一种新的诊断工具。

排序理由这是一篇介绍新评估指标的学术论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

报道来源 [2]

arXiv cs.LG TIER_1 English(EN) · Alessandro Gambetti, Qiwei Han, Cl\'audia Soares, Hong Shen · 2026-06-04 04:00

Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

arXiv:2606.04613v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) struggle when applied to medical image-text data, yet the tools available to diagnose this failure remain limited. Existing representation alignment metrics are symmetric, collapsing both modalities i…
Hugging Face Daily Papers TIER_1 English(EN) · 2026-06-03 08:50

Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

Vision-Language Models (VLMs) struggle when applied to medical image-text data, yet the tools available to diagnose this failure remain limited. Existing representation alignment metrics are symmetric, collapsing both modalities into a single score and hiding which modality drive…

报道来源 [2]

Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

相关实体

相关话题