English(EN)LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs’ Probabilistic Beliefs
研究发现 LLMs 在贝叶斯推理和信念更新方面存在困难 · 跟踪 4 个来源
作者PulseAugur 编辑部·[5 个来源]·
近期研究表明，大型语言模型（LLMs）在一致地表示和更新概率信念方面存在困难，而这对于复杂领域中的决策至关重要。研究发现，LLMs 在处理证据时常常表现出内部不一致性，有时非贝叶斯启发式方法在下游任务中的表现优于精确的贝叶斯更新。这表明 LLMs 的内部世界模型可能被错误指定，导致认知短视和校准不当，尤其是在处理长尾事实或隐藏信息时。
AI
Modern AI systems are being deployed in complex domains such as medicine, science, and law, where there is often not a single correct answer given the observed evidence. Such systems must be able to represent and update uncertain beliefs about the world as new evidence arrives to…
arXiv cs.CL
TIER_1English(EN)·Zhuoshi Pan, Junru Lu, Yan Qian, H. Vicky Zhao, Di Yin, Xing Sun·
arXiv:2608.28478v1 Announce Type: new Abstract: Factual question answering (QA) typically assumes a single canonical answer, obscuring whether large language models (LLMs) retain divergent accounts of long-tail facts. To address this gap, we introduce ElephantBench, a closed-book…
arXiv:2608.26887v1 Announce Type: new Abstract: LLMs are thought to track "belief states," i.e., running probability distributions over the latent variables that govern language (Shai et al., 2024; Sarfati et al., 2026), but so far this has only been comprehensively demonstrated …
ElephantBench evaluates whether large language models retain multiple conflicting accounts of long-tail facts through a graph-based pipeline that creates verifiable multi-account questions, revealing widespread epistemic incompleteness.
Agentic systems increasingly gate actions on a model's own stated confidence, which assumes confidence tracks correctness at the moment of acting. We test this in a hidden-information chess variant where royal status can be secretly, repeatedly relocated between pieces, and where…
