New XSPA attack method targets Vision-Language Models

Researchers have developed XSPA, a novel method for creating adversarial perturbations on Vision-Language Models (VLMs). This technique crafts imperceptible X-shaped sparse perturbations that can significantly degrade VLM performance across multiple tasks, including zero-shot classification, image captioning, and visual question answering. While XSPA reduces accuracy on tasks like COCO, it serves as a controlled stress test for VLM robustness rather than a universally superior attack method. AI

IMPACT This research highlights potential vulnerabilities in Vision-Language Models, prompting further investigation into their robustness and security.

RANK_REASON The cluster contains an academic paper detailing a new method for attacking Vision-Language Models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• COCO
• GPT-4
• Jiaju Han
• OpenAI CLIP ViT-L/14
• OpenCLIP ViT-B/16
• Vision--Language Models

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →