New metric evaluates lexical stress in English-to-Chinese S2ST

Researchers have developed a new method to evaluate and preserve lexical stress in English-to-Chinese speech-to-speech translation (S2ST). They created a stress-annotated Chinese dataset and a Mandarin stress detector using XLS-R, integrating it with the English EmphAssess system to propose a novel objective metric for cross-lingual stress evaluation. A fine-tuned CosyVoice3 system demonstrated improved stress translation capabilities while maintaining translation quality, and the new evaluation metric showed strong correlation with human judgment. AI

IMPACT This research could improve the naturalness and expressiveness of speech-to-speech translation systems.

RANK_REASON The cluster contains an academic paper detailing a new method for speech-to-speech translation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• CosyVoice3
• EmphAssess
• English
• Standard Chinese
• XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →