New metric detects and reduces LLM legal citation hallucinations

Researchers have developed a new metric called Citation Grounding (CG) to detect and reduce hallucinations in Large Language Models (LLMs) when generating legal citations. This metric, tested against a large dataset of Ukrainian court decisions, breaks down hallucinations into precision, relevance, and temporality issues. To address these issues without human annotation, they also introduced Citation Grounding DPO (CG-DPO), a method that fine-tunes LLMs using algorithmically generated preference pairs, achieving high accuracy in distinguishing correct from corrupted citations. AI

IMPACT Introduces a novel evaluation framework for LLM legal citation accuracy, potentially improving reliability in legal AI applications.

RANK_REASON Academic paper introducing a new metric and method for evaluating LLM safety. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Amazon Nova Pro/Lite
• Citation Grounding
• Citation Grounding DPO
• Claude Haiku 4.5
• Large Language Models
• Mistral Pixtral Large
• Qwen2.5-7B-Instruct
• Ukrainian court decisions

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →