Urdu fake news detection hindered by dataset length confound

Researchers have conducted a study on Urdu fake news detection, highlighting significant challenges in cross-dataset generalization. Using the XLM-RoBERTa model and two distinct Urdu datasets, the study found that while transfer from the Notri-Fact dataset to the Ax-to-Grind corpus yielded a respectable F1 score of 0.771, the reverse transfer resulted in a near-complete collapse, achieving an F1 score of 0.005. This drastic performance drop was attributed to a length confound in the Ax-to-Grind dataset, where fake news articles were substantially longer than real ones, leading to shortcut learning by the model. The study proposes a diagnostic methodology to identify such confound-driven behavior in multilingual fake news detection. AI

IMPACT Highlights the critical need for robust cross-dataset generalization in NLP models, particularly for under-resourced languages, to prevent shortcut learning.

RANK_REASON Academic paper detailing a new empirical study on fake news detection. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Ax-to-Grind Urdu corpus
• Logistic Regression
• Notri-Fact Urdu dataset
• Urdu
• XLM-RoBERTa

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →