New framework boosts speech translation robustness by transferring text-based adversarial training

Researchers have developed a new framework called Cross-Modal Robustness Transfer (CMRT) to improve the robustness of End-to-End Speech Translation (E2E-ST) models. Current E2E-ST models are often trained on clean datasets and struggle with variations found in real-world speech, such as dialectal differences or non-native speech. CMRT addresses this by transferring adversarial robustness from text-based training to speech translation, eliminating the need for computationally expensive adversarial speech data generation. Experiments show CMRT improves adversarial robustness by over 3 BLEU points across four language pairs. AI

IMPACT Enhances the reliability of speech translation systems in real-world, noisy conditions.

RANK_REASON The cluster contains an academic paper detailing a new methodology for improving AI model robustness. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Abderrahmane Issam
• alphaXiv
• arXiv
• CatalyzeX
• Cross-Modal Robustness Transfer
• DagsHub
• E2E-ST
• End-to-End Speech Translation
• Gotit.pub
• Hugging Face
• ScienceCast

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →