Researchers have developed LEVANTE-bench, a new benchmark designed to compare the cognitive abilities of vision-language models (VLMs) with those of children. The benchmark utilizes tasks and data from the LEVANTE project, assessing VLMs against 1,547 children aged 5-12 across three countries. Findings indicate that while more capable VLMs align better with children's performance on tasks and items, their error patterns do not consistently match human children's, with smaller models sometimes better reflecting younger children's mistakes. Notably, even top-performing VLMs struggled with complex reasoning tasks like matrix reasoning and mental rotation, suggesting current VLM architectures only partially mirror human cognitive development. AI
IMPACT Introduces a novel method for evaluating VLM cognitive alignment with human development, potentially guiding future model improvements.
RANK_REASON The cluster contains an academic paper detailing a new benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
