New TrendFact benchmark evaluates LLM fact-checking hotspot perception

Researchers have introduced TrendFact, a new benchmark designed to evaluate the Hotspot Perception Ability (HPA) of Automatic Fact-Checking (AFC) systems, particularly Large Language Models (LLMs) and Reasoning Large Language Models (RLMs). The benchmark addresses a critical risk asymmetry challenge faced by these systems in real-world, resource-constrained environments. TrendFact includes over 7,600 samples and proposes novel metrics like the Explanation Consistency Score (ECS) and Hotspot Claim Perception Index (HCPI) to assess HPA and reasoning reliability. Experiments show that current AFC systems perform poorly on TrendFact, but a proposed FactISR framework demonstrates effectiveness in improving HPA and computational efficiency for RLMs-served AFC systems. AI

IMPACT This benchmark could lead to more robust and efficient AI fact-checking systems capable of prioritizing information based on social impact.

RANK_REASON The cluster describes a new academic paper introducing a benchmark and framework for evaluating LLM fact-checking capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• FactISR
• Large Language Models
• Reasoning Large Language Models
• TrendFact
• Xiaocheng Zhang

• paper
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →