Vision Encoders Show Weak Alignment with Human Color Perception

A new study published on arXiv investigates whether deep vision encoders, commonly used in computer vision tasks, exhibit human-like color discrimination thresholds. Researchers compared over 50 pre-trained vision encoders, including convolutional networks and vision transformers, against human perceptual thresholds using controlled chromatic stimuli and a region-overlap metric (mIoU). The findings indicate a generally weak alignment between model representations and human thresholds, with the best models achieving an mIoU below 0.25. Self-supervised encoders performed better than supervised ones, while language-supervised models showed highly varied results. AI

IMPACT Suggests current large-scale visual training objectives may not naturally lead to human-like chromatic sensitivity in AI models.

RANK_REASON The cluster contains a research paper detailing an exploratory study on AI model capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Deep Neural Networks
• Deep Vision Encoders
• foundation model
• Language-supervised Models
• Self-supervised Encoders
• Supervised Encoders
• Vision Encoders
• Vision Transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →