New LLM-generated dataset boosts Arabic fact-checking capabilities

Researchers have introduced Arafa, a new large-scale dataset designed for fact-checking in Modern Standard Arabic. This dataset was created using a three-step automated framework that leverages large language models (LLMs) to generate claims and supporting or refuting evidence from Arabic Wikipedia. The dataset contains over 181,000 claim-evidence pairs, and initial evaluations show strong inter-annotator agreement and high accuracy in automatic validation. Fine-tuning transformer-based models on Arafa has yielded promising results, with the best model achieving a Macro F1-score of 77% on test data, highlighting its utility for advancing Arabic natural language processing in fact-checking. AI

IMPACT This dataset could significantly improve the accuracy and availability of fact-checking tools for Arabic content.

RANK_REASON The cluster describes a new academic paper introducing a dataset for a specific NLP task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Arabic Wikipedia
• Arafa
• Christophe Khalil
• Cohen's kappa
• Hugging Face
• Macro F1-score
• Modern Standard Arabic

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →