New ASR framework enhances multilingual recognition with speech and context alignment

Researchers have developed a new multilingual automatic speech recognition (ASR) framework that supports various languages and accents by integrating speech and contextual information. The system uses a frozen speech encoder and a decoder-only language model, enhanced by a lightweight projection module for structured context prompts like dialogue history. A contrastive learning objective aligns speech and context representations in a shared embedding space, leading to performance gains of over 5% on real-world conversational speech across 11 languages and 5 English dialects. AI

IMPACT This research could lead to more robust and versatile speech recognition systems, improving accessibility and usability across different languages and conversational contexts.

RANK_REASON This is a research paper detailing a new framework for multilingual ASR. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• ScienceCast
• Yuchen Zhang

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →