New benchmark reveals 60% of VLMs can infer private data

Researchers have developed MultiPriv, a new benchmark to assess the individual-level privacy reasoning capabilities of vision-language models (VLMs). The benchmark includes a bilingual multimodal dataset designed to link identifiers like faces and names to sensitive attributes, enabling tasks such as attribute detection and chained inference. Initial evaluations show that 60% of tested VLMs can perform individual-level privacy reasoning with up to 80% accuracy, highlighting a significant privacy risk. AI

IMPACT Highlights significant privacy risks in VLMs, potentially influencing future model development and data handling practices.

RANK_REASON The cluster contains a research paper introducing a new benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• MultiPriv
• Vision-Language Models
• Xiongtao Sun

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →