AI models struggle with legal citation accuracy; retrieval is key

A new research paper explores the effectiveness of different AI approaches for accurately citing legal statutes, specifically focusing on the Ontario Residential Tenancies Act. The study compared a base model, a fine-tuned model, a retrieval-augmented generation (RAG) model, and a hybrid SFT+RAG model using the Qwen2.5-7B-Instruct model. Results indicate that retrieval methods are crucial for eliminating hallucinations and achieving accurate citations, with the SFT+RAG hybrid model achieving the highest exact-match score of 0.481. AI

IMPACT Highlights the necessity of retrieval-augmented generation for accurate legal citation by AI models.

RANK_REASON Research paper evaluating AI model performance on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Ontario Residential Tenancies Act
• Qwen2.5-7B-Instruct

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →