A new study published on arXiv investigates how large language models (LLMs) revise their decisions when faced with expert disagreement. The research found that LLMs often fail to improve accuracy when revising answers under expert disagreement, with accuracy sometimes dropping significantly. The study highlights that LLMs are influenced by the prestige of the disagreeing source, leading to more revisions even when the expert's opinion is incorrect. The authors propose that revision pipelines should be evaluated using paired, conditional metrics specific to each model and protocol. AI
IMPACT This research suggests that current LLM revision mechanisms are not robust to expert disagreement, potentially impacting their reliability in critical decision-making scenarios.
RANK_REASON The cluster contains a single academic paper detailing research findings. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
