MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models
Researchers have developed MultiPriv, a new benchmark to assess the individual-level privacy reasoning capabilities of vision-language models (VLMs). The benchmark includes a bilingual multimodal dataset designed to link identifiers like faces and names to sensitive attributes, enabling tasks such as attribute detection and chained inference. Initial evaluations show that 60% of tested VLMs can perform individual-level privacy reasoning with up to 80% accuracy, highlighting a significant privacy risk. AI
IMPACT Highlights significant privacy risks in VLMs, potentially influencing future model development and data handling practices.