New HPRO framework enhances emotional expressiveness in TTS models

Researchers have introduced HPRO, a hierarchical progressive reward optimization framework designed to improve emotional expressiveness in large language model-based text-to-speech (TTS) systems. This new framework addresses issues like information conflict and scale gap found in existing preference-driven optimization methods. HPRO utilizes an HD-Emo codec to separate content and emotional preference tokens, thereby isolating emotional optimization from semantic content and mitigating reward hacking. The system progressively aligns objectives at frame, word, and sentence levels, leading to enhanced emotional expression while maintaining linguistic intelligibility. AI

IMPACT This research could lead to more emotionally nuanced and natural-sounding AI-generated speech, improving user experience in applications like virtual assistants and audio content creation.

RANK_REASON The cluster contains an academic paper detailing a new method for improving text-to-speech models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• HD-Emo codec
• Hugging Face
• large language model
• text-to-speech
• supervised fine-tuning

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →