New method enables voice cloning in text-to-audio-video models

Researchers have developed a method to add voice cloning capabilities to text-to-audio-video (T2AV) generation models. By incorporating a single, zero-initialized linear layer and fine-tuning, T2AV models can be adapted to clone voices from short reference recordings. This approach significantly outperforms existing voice-cloning text-to-speech baselines in speaker-encoder cosine similarity. AI

IMPACT Enables more personalized and realistic audio-visual content generation by allowing custom voice integration.

RANK_REASON The cluster contains an academic paper detailing a new method for voice cloning in AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• ECAPA-TDNN
• Gotit.pub
• Hugging Face
• Resemblyzer
• ScienceCast
• Viacheslav Vasilev
• WavLM-SV

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →