Researchers have identified a phenomenon called "procedural hallucination" in language models, where models generate answers that are supported by retrieved context but conflict with the model's own skills or procedures. To study this, they created a benchmark called SourceClash, comprising 1,271 bundles and 6,355 prompts across ten procedural domains. Across nine tested models, the rate of valid answers dropped significantly to an average of 25.7% when retrieval context conflicted with a skill, compared to over 70% when the context was supportive. Analysis suggests that interventions targeting specific value pathways within the model can improve the accuracy of its final answers. AI
IMPACT Identifies a specific failure mode in LLMs related to conflicting information, potentially guiding future model development and evaluation.
RANK_REASON Research paper detailing a new benchmark and analysis of language model behavior. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
