New framework evaluates physical adversarial robustness of edge VLMs

Researchers have developed a new framework to evaluate the physical adversarial robustness of edge Vision-Language Models (VLMs). This framework introduces a metric called conditional attack success rate (cASR) to compare VLMs with traditional object detectors. The study found that VLMs can be as vulnerable as object detectors to adversarial attacks, with failure modes often depending on the model's output format rather than its architecture. Notably, VLMs can shift to more dangerous failure modes like displacement and substitution under distribution shifts. AI

IMPACT This research highlights potential security vulnerabilities in edge VLMs, crucial for developers deploying these models in safety-critical applications.

RANK_REASON Research paper introducing a new evaluation framework and metric for VLM robustness. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• conditional attack success rate
• Hugging Face
• Object Detectors
• vision-language model
• Vision--Language Models

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →