New 'PRAC' attack targets vision modality of AI agents

Researchers have developed a new attack called PRAC that targets the vision modality of multimodal foundation models, specifically Computer Use Agents (CUAs). Unlike previous attacks that directly manipulated model output, PRAC redirects the model's attention using a stealthy adversarial patch. This method has been demonstrated to successfully manipulate a CUA on an online shopping platform to select a specific target product. While the attack requires white-box access for creation, it shows generalization to fine-tuned models, posing a significant security risk for CUAs built on open-weight models. AI

IMPACT This research highlights a critical security vulnerability in vision-based AI agents, potentially impacting the safety and reliability of autonomous systems interacting with graphical interfaces.

RANK_REASON Research paper detailing a novel attack method on AI agents. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX
• Computer Use Agents
• DagsHub
• Dominik Seip
• Gotit.pub
• Hugging Face
• IArxiv
• multimodal foundation models
• ScienceCast
• vision modality

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →