AI system tackles Arabic Islamic Q&A hallucinations with 92.8% detection accuracy

Researchers have developed a system for the HalluScoring 2026 Task 2.1, which focuses on detecting hallucinations and identifying verified answers in Arabic Islamic question answering. The system, built upon a fine-tuned google/gemma-4-12B-it model, achieved a Macro-F1 score of 0.928 for hallucination detection and an option accuracy of 0.895 for answer selection. While effective at identifying factual errors, the system's performance indicates that distinguishing the correct answer from plausible incorrect options remains a greater challenge. AI

IMPACT Improves accuracy in specialized Arabic Q&A systems, highlighting challenges in distinguishing verified answers from plausible falsehoods.

RANK_REASON Academic paper detailing a new system for detecting hallucinations in LLM responses for a specific domain. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• google/gemma-4-12B-it
• HalluScoring 2026
• Hugging Face

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →