New benchmark reveals multilingual safety gaps in vision-language models

Researchers have developed MLingualFC, a new multilingual benchmark to test the safety vulnerabilities of vision-language models (VLMs). This benchmark uses flowchart images encoded with harmful instructions in five languages: Hindi, Punjabi, Spanish, Romanian, and German. Evaluations of models like Qwen2.5-VL, Gemma-4, and Pangea revealed that visual attacks are highly successful in Latin-script languages, indicating current safety measures do not generalize well across languages and modalities. AI

IMPACT Highlights the need for more robust, multilingual safety alignment in advanced AI models.

RANK_REASON The cluster contains an academic paper introducing a new benchmark for evaluating AI model safety. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Gemma-4
• MLingualFC
• Pangea
• Qwen2.5-VL

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →