New benchmark evaluates cross-modal knowledge unlearning in vision-language models

Researchers have introduced UNLINK-VL, a new benchmark designed to evaluate the effectiveness of knowledge unlearning in vision-language models (VLMs). This benchmark addresses the gap in current research, which primarily focuses on unlearning within single modalities, by assessing how well knowledge removal transfers across text and visual domains. Experiments show that multimodal unlearning is effective when evaluated visually, but text-only unlearning transfers poorly to visual and cross-modal tasks, suggesting that intra-modal evaluations may overestimate unlearning effectiveness. AI

IMPACT Highlights the need for cross-modal evaluation in AI unlearning to accurately assess trustworthiness.

RANK_REASON The cluster contains an academic paper introducing a new benchmark for evaluating AI model capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Large Language Models
• UNLINK-VL
• vision-language model
• Wikidata

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →