New game-based framework reveals AI color perception biases

Researchers have developed a new evaluation framework using the game Hues and Cues to assess the perceptual alignment between Contrastive Vision-Language Models (CVLMs) and humans. This framework maps color cells to a chromaticity diagram to calculate perceptual distances for a 100-word vocabulary. The study found that while CVLMs can replicate human biases for concrete concepts, they diverge in abstract domains, exhibiting semantic misclassification or a collapse into a default blue coordinate. The findings suggest that curated pre-training datasets are more effective than massive, uncurated ones in reducing these misalignments, highlighting that current CVLMs still struggle to capture nuanced human color memory. AI

IMPACT Highlights limitations in current AI's ability to grasp nuanced human perception, suggesting a need for better training data and evaluation methods.

RANK_REASON The cluster contains an academic paper detailing a new evaluation methodology for AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Contrastive Vision-Language Models
• Hues and Cues
• Nuria Alabau-Bosque

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →