Language models' linguistic representations predict human brain visual perception

A new study published on arXiv explores how linguistic representations from language models can effectively model high-level visual perception in the human brain. Researchers found that machine-generated captions, when processed by text embedders, often outperformed human-annotated captions in predicting brain responses to images. The study also indicated that intermediate network depths in language models are optimal for capturing both brain and behavioral alignment with visual similarity judgments. AI

IMPACT Suggests that advanced language models can serve as valuable tools for understanding human visual processing.

RANK_REASON Academic paper detailing research findings on language models and visual perception. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Autoregressive LMs
• Hugging Face
• human brain
• Language Models
• Text Embedders

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →