Researchers have introduced Omnilingual-MsASR, a new benchmark designed to evaluate the capabilities of omni-modal large language models (LLMs) in handling long-form, multi-speaker audio and video content. The benchmark covers six languages and recordings up to 30 minutes, featuring scenarios with up to 20 speakers. Initial evaluations of leading omni-modal LLMs demonstrated significant performance limitations, with many models exceeding 100% word error rates, highlighting challenges in temporal localization, speaker disambiguation, and cross-modal synergy. AI
IMPACT Highlights key challenges for future omni-modal foundation models in speech recognition and speaker identification.
RANK_REASON The cluster describes a new academic benchmark for evaluating LLMs, presented in a research paper on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- DagsHub
- Gotit.pub
- Hugging Face
- MsASR
- Omni Large Language Models
- Omnilingual-MsASR
- ScienceCast
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →