Hugging Face has developed a new benchmark and dataset to evaluate how automatic speech recognition (ASR) systems handle code-switched speech, a common practice among bilingual individuals. The benchmark focuses on four language pairs relevant to enterprise customer bases: Spanish-English, French-English, Canadian French-English, and German-English. The study reports results from seven ASR systems, with ElevenLabs Scribe V2, Gemini 3 Flash, and Assembly AI Universal 3-Pro emerging as top performers across various metrics. AI
IMPACT This benchmark will help improve voice agent performance for bilingual customer bases, leading to better user experiences and operational efficiency.
RANK_REASON The cluster describes a new benchmark and dataset for evaluating ASR systems on code-switched speech, including methodology and results. [lever_c_demoted from research: ic=1 ai=1.0]
- Assembly AI Universal 3-Pro
- ElevenLabs Multilingual V2
- ElevenLabs Scribe V2
- Gemini 3 Flash
- GPT-5
- Hugging Face
- OpenAI
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →