Researchers have developed a new geometric measure to quantify coarticulation, the influence of surrounding sounds on speech articulation, for speech-driven 3D facial animation. This measure, called lip-path length, compares the actual motion trajectory to the shortest path between phonetic positions. When applied to four leading animation methods—DiffPoseTalk, ARTalk, FaceFormer, and CodeTalker—all were found to simplify lip trajectories compared to real speech, with three showing significant deficits. A viewer study with over 3,500 judgments confirmed that viewers prefer real speech and penalize exaggerated or overly simplified lip motion, highlighting a key area for improvement in synthesized articulation. AI
IMPACT Identifies a specific perceptual deficit in speech-driven animation, providing a target for future model development.
RANK_REASON Academic paper introducing a new methodology and evaluation. [lever_c_demoted from research: ic=1 ai=1.0]
- ARTalk
- arXiv
- DiffPoseTalk
- FaceFormer
- Przemyslaw Musialski
- The Shape of Speech: A Geometric Measure of Coarticulation for Speech-Driven 3D Facial Animation
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →