A new study published on arXiv investigated the ability of six frontier language models to correctly attribute answers to different age groups. The models, including Claude Fable-5, Gemini-3.1 Pro, and GPT-5.6 Sol, were tested on their capacity to match exam answers to the ages of the authors (17, 28, and 45). Researchers found that most models struggled with this task, often confidently misattributing the youngest author's answer to an adult and performing at or near chance levels. Confidence in their predictions was frequently uncorrelated or inversely correlated with accuracy, and even after being shown the correct answers, one model continued to exhibit poor performance while articulating its own biases. AI
IMPACT This research highlights potential biases in frontier language models and their miscalibrated confidence, suggesting limitations in their nuanced understanding and application of social context.
RANK_REASON The cluster is based on a research paper published on arXiv detailing a behavioral study of language models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
