New VLM Vulnerability: Token Pruning Amplifies Malicious Content

Researchers have identified a new vulnerability in vision-language models (VLMs) related to token-pruning techniques, which are used to accelerate model performance by removing redundant visual tokens. This vulnerability, termed Pruning-Induced Malicious Amplification, can inadvertently amplify toxic semantics by causing the model's attention to focus on malicious foreground tokens after benign background tokens are removed. To combat this, a new plug-and-play mechanism called Safety-Aware Pruning (SAP) has been developed. SAP works at inference time to identify malicious anchors, restore benign tokens, and reallocate attention, demonstrating a significant reduction in adversarial success rates without sacrificing efficiency or utility. AI

IMPACT Identifies a new class of vulnerabilities in VLMs related to optimization techniques, potentially impacting the safe deployment of multimodal AI systems.

RANK_REASON Academic paper detailing a new vulnerability and mitigation strategy for VLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Hugging Face
• Pruning-Induced Malicious Amplification
• Query-based Compression
• Safety-Aware Pruning
• Token Pruning
• vision-language model

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →