A position paper argues that current AI leaderboards are not serving the Global South due to a lack of independent governance and mechanisms for metric evolution. Despite the existence of high-quality regional benchmarks for languages like Hindi, Swahili, and Arabic, these are not incorporated into global leaderboards. The paper uses India as a case study, highlighting that AI practitioners prefer formal governance and disclosure-based conflict management over simply more data. The proposed solution is the development of regional leaderboards with independent governance from their inception. AI
IMPACT Current AI evaluation frameworks may perpetuate biases against non-Western languages and regions, necessitating the development of more inclusive and independently governed benchmarks.
RANK_REASON The item is an academic position paper discussing institutional design flaws in AI leaderboards and proposing solutions. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
