Hugging Face has expanded its Open ASR Leaderboard by introducing two new evaluation sets, Monsoon en-IN and Monsoon hi-IN, to better represent languages from the Global South. These additions aim to address the known issue of automated speech recognition (ASR) systems performing poorly for certain demographics, which is not captured by traditional word error rate (WER) metrics. The new datasets are designed to vary across multiple axes including geography, age, gender, devices, and acoustic environments, providing a more comprehensive evaluation of ASR models. AI
IMPACT Improves evaluation of ASR models for underrepresented linguistic and demographic groups.
RANK_REASON Addition of new datasets to an existing benchmark for evaluating ASR models. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
