New AfriEconQA benchmark challenges AI on economic report reasoning

Researchers have introduced AfriEconQA, a new benchmark designed to test the quantitative and temporal reasoning capabilities of AI models when processing lengthy institutional documents. This benchmark, derived from 220 World Bank economic reports on African economies, includes 4,309 question-answer instances focused on precise numerical and temporal data extraction. Evaluations of models like Qwen 3.6 35B, DeepSeek V4-Pro, and Gemma 4 12B IT using retrieval-augmented generation showed significant gains but still struggled to achieve high accuracy, indicating a challenging open problem for AI in handling complex economic reports. AI

IMPACT Highlights limitations in current LLMs for precise quantitative and temporal reasoning over long, complex documents, indicating areas for future research and development.

RANK_REASON The cluster describes a new academic benchmark for evaluating AI models on a specific task, presented in a research paper. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• AfriEconQA
• arXiv
• DeepSeek V4-Pro
• Edward Ajayi
• Gemma-4-12B-it
• Hugging Face
• Qwen 3.6:35B
• World Bank

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →