New framework enables LLMs to abstain from fact-checking weak evidence

Researchers have developed a new framework called Evidence Chain Evaluation (ECE) to improve the reliability of large language models in fact-checking. ECE allows models to abstain from making a decision when evidence is weak or inconsistent, instead of forcing a binary true/false verdict. This selective fact-checking approach uses a tool-using verification agent that gathers evidence from various sources, including web searches and executable checks. When tested on the ECE-Bench dataset, ECE demonstrated high accuracy on answered claims and effectively deferred cases with lower-reliability evidence, functioning as a safety mechanism. AI

IMPACT Enhances LLM reliability in fact-checking by allowing abstention on uncertain claims, improving safety and trustworthiness.

RANK_REASON Research paper detailing a new framework for LLM fact-checking. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• ECE
• ECE-Bench
• Evidence Chain Evaluation
• Gotit.pub
• Hugging Face
• Influence Flower
• ScienceCast

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →