VLMs have diffuse visual safety neurons compared to localized text safety

A new research paper explores the safety mechanisms within vision-language models (VLMs), investigating how visual inputs can lead to harmful outputs even when text-based inputs are refused. The study proposes a novel two-stage detection pipeline with iterative ablation and introduces benchmarks like ViSafe-Detect and ViSafe-Eval to isolate visual and textual safety signals. Findings indicate that text safety in VLMs is highly localized, with a small percentage of neurons responsible for refusals, whereas visual safety is more diffuse and high-dimensional, requiring a larger number of neurons. AI

IMPACT This research could lead to more robust safety alignment for multimodal AI systems, addressing a key vulnerability in current models.

RANK_REASON Research paper published on arXiv detailing novel methods for analyzing VLM safety mechanisms. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv
• ScienceCast
• ViSafe-Detect
• ViSafe-Eval
• Vision--Language Models

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →