New framework audits multimodal LLMs by identifying decision-driving modalities

Researchers have developed a new framework called Counterfactual Modality Attribution (CMA) to assess which modality, such as images or text, is primarily responsible for a multimodal large language model's (MLLM) prediction. This method addresses the issue where MLLMs might produce correct outputs by relying on incorrect evidence, masking potential shortcut learning or unsafe reasoning. CMA utilizes coupled diffusion priors to generate counterfactuals and applies Shapley values to quantify modality-level contributions, demonstrating high accuracy in identifying the decision-driving modality in controlled tests and outperforming existing methods on real-world clinical data. AI

IMPACT Enhances auditing of multimodal LLMs, crucial for safety-critical applications by revealing hidden reasoning failures.

RANK_REASON The cluster contains a research paper detailing a new framework for multimodal LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• Competition and Markets Authority
• Counterfactual Modality Attribution
• Hugging Face
• Multimodal large language models
• Shapley values

• paper
• safety
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →