A new evaluation protocol called Knowledge Answer Divergence (KAD) has been introduced to assess how language models handle factual inaccuracies after an initial false correction. This method distinguishes between a model's initial adoption of false information and its subsequent persistence, recovery, or abstention. Experiments with models like Llama 3.2 Instruct 3B and Qwen 3.5-4B showed that conversational history and specific wording significantly impact a model's factual reliability. AI
IMPACT Introduces a new method for evaluating and potentially improving the factual accuracy and robustness of language models.
RANK_REASON Research paper introducing a new evaluation protocol for language models. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- Hugging Face
- Knowledge After Collapse: Context and Wording Shape Factual Accessibility in Language Models
- Knowledge Answer Divergence
- Llama 3.2 Instruct 3B
- Qwen 3.5-4B
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →