A new research paper explores how large language models (LLMs) update their probability estimates based on new information. The study found that LLMs generally adjust their probabilities in the correct direction when presented with evidence, but this adjustment is slightly reduced when the information is reported rather than directly stated. Furthermore, using multiple weighted estimates instead of a single probability can decrease the model's sensitivity to evidence, particularly in smaller, locally run models. AI
IMPACT Investigating how LLMs process information is crucial for understanding their reliability as assistants and in experimental settings.
RANK_REASON Research paper on LLM behavior published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →