DPO improves code-switching speech recognition in audio LLMs

Researchers have developed a new method using Direct Preference Optimization (DPO) to improve how audio large language models handle speech that switches between English and Mandarin. The models often fail by omitting languages, translating instead of transcribing, or hallucinating content. By training on 100,000 preference pairs, the models learned to preserve the mixed-language content, significantly reducing transcription errors. AI

IMPACT Enhances the accuracy of multilingual speech recognition in LLMs, potentially improving global accessibility and usability.

RANK_REASON Academic paper detailing a new method for improving LLM performance on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Audio LLMs
• Direct Preference Optimization
• English-Mandarin

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →