Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Models
A new benchmark called AuthorityBench, comprising 220,564 prompts across general knowledge, science, law, and medicine, has been developed to study how citation presence influences large language models' behavior. The research found that the presence of citations, even fabricated ones, consistently increases hallucination rates compared to prompts without citations. This effect is most pronounced when false citations accompany true claims, significantly raising hallucination rates, particularly in the general knowledge domain. AI
IMPACT This research highlights a critical vulnerability in LLMs, suggesting that citation-augmented systems may require significant re-evaluation to mitigate increased hallucination rates.