Researchers have developed Gaspard, a novel unified model for generating both speech and co-speech gestures. Unlike previous cascaded systems, Gaspard integrates speech and gesture generation into a single autoregressive flow-matching framework. This approach, trained on the Seamless Interaction dataset, allows for better synchronization and expressivity by decoupling classifier-free guidance for each modality. Evaluations indicate Gaspard outperforms unimodal baselines in gesture generation and produces more synchronous speech and gestures than cascaded systems. AI
IMPACT This unified approach to speech and gesture synthesis could lead to more natural and expressive AI-powered communication tools.
RANK_REASON The cluster contains an academic paper detailing a new model for multimodal generation. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →