Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Models
Researchers have developed Ex-Omni, an open-source model designed to integrate 3D facial animation generation with omni-modal large language models (OLLMs). This model addresses the challenge of bridging LLMs' discrete reasoning with the continuous dynamics of facial motion by using speech units for temporal structure and hidden speech representations for facial cues. Ex-Omni aims to improve human-computer interaction by enabling OLLMs to produce synchronized speech and 3D facial animations, demonstrating faster generation and better audio-visual synchronization compared to existing cascaded methods. AI
IMPACT Enables more natural human-computer interaction by synchronizing LLM-generated speech with 3D facial animations.