Researchers have introduced MuSeToM, a new benchmark designed to evaluate the belief revision capabilities of large language models (LLMs) in the context of Theory of Mind (ToM). This benchmark, comprising 2,415 instances derived from BigToM and OpenToM, assesses whether LLMs can determine when perceptual evidence warrants a change in belief and infer the updated belief state. Evaluations using MuSeToM revealed that current state-of-the-art LLMs can often detect potential belief changes but struggle to accurately identify the revised belief, sometimes overemphasizing perceptually salient but epistemically weak cues. AI
IMPACT This benchmark could drive improvements in LLMs' social reasoning and ability to understand nuanced belief updates based on evidence.
RANK_REASON The cluster describes a new academic paper introducing a novel benchmark for evaluating LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
