New method CODE improves LLM knowledge editing by reducing self-refutation

A new research paper introduces CODE (Causal On-policy Distillation for Editing), a method designed to improve knowledge editing in large language models. Traditional methods, which overwrite facts directly, can lead to "Epistemic Dissonance," causing models to contradict new information. CODE addresses this by grounding updates in causal narratives, significantly reducing self-refutation rates from 95.6% to as low as 1.8% in experiments with LLaMA-3.1 and Qwen-2.5, while maintaining high multi-hop accuracy. AI

IMPACT This research could lead to more reliable and coherent updates for LLMs, improving their ability to integrate new information without internal contradictions.

RANK_REASON The cluster contains a research paper detailing a new method for knowledge editing in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• CODE
• LLaMA-3.1
• Qwen-2.5

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →