Study Compares Speech Synthesis Content Representations

A new study published on arXiv explores various content representations for speech synthesis, comparing their effectiveness in voice conversion, speech-to-speech translation, and multimodal language models. Researchers trained generative models conditioned solely on different representations, including SSL features, supervised tokens, posteriorgrams, and neural audio codecs. The findings indicate that some representations excel at reconstructing original audio, while others effectively disentangle speaker identity, suggesting that information capacity and training objectives play crucial roles in achieving this separation. AI

IMPACT Provides insights into optimizing speech synthesis models by understanding the strengths of different content representations.

RANK_REASON Research paper published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• A Comprehensive Study of Content Representations for Speech Synthesis
• arXiv
• Diego Torres Guarin
• Hugging Face
• Neural Audio Codecs
• posteriorgrams
• SSL features
• supervised tokens

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →