Textual personas fail to make vision-language models see like people

A new study published on arXiv investigates the effectiveness of textual personas in guiding vision-language models (VLMs). Researchers found that instructing VLMs with personas like "You have low vision" does not make them perceive or answer like the individuals they are meant to represent. The models' responses, even when prompted with personas, did not accurately reflect the experiences of people with low vision, color blindness, or hearing loss, and often failed to outperform simple rule-based systems. This suggests that while VLMs can estimate the effects of visual impairments on humans, they do not genuinely "see" or process information as those individuals would. AI

IMPACT This research highlights a current limitation in how vision-language models interpret and respond to user personas, suggesting a gap between simulated understanding and genuine perception.

RANK_REASON The cluster contains a research paper detailing findings about the limitations of vision-language models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• color blindness
• CORE Recommender
• DagsHub
• Gotit.pub
• hearing loss
• Hugging Face
• Influence Flower
• low vision
• Open Models
• ScienceCast
• vision-language model

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →