Researchers have developed a new method for evaluating retrieval systems specifically for Arabic Islamic jurisprudence (fiqh). This approach isolates retrieval performance from generation errors, allowing for a more precise assessment of how well systems can find relevant passages. The study introduces a test collection for Arabic fiqh and compares various retrieval strategies, with the best performing system achieving an MRR@5 of 0.524, which improved to 0.553 with fine-tuning. A key challenge identified is differentiating passages that contain the specific ruling from those that are merely topically similar. AI
IMPACT This research could lead to more accurate AI systems for understanding and querying specialized legal and religious texts.
RANK_REASON The item is an academic paper detailing a new benchmark and evaluation methodology for a specific domain. [lever_c_demoted from research: ic=1 ai=1.0]
Read on arXiv cs.IR (Information Retrieval) →
- Arabic fiqh
- Arabic Islamic Jurisprudence
- arXiv
- Hugging Face
- MRR@5
- retrieval-augmented generation
- Somaya Eltanbouly
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
