New framework EmoteGPT generates 3D facial expressions from text

Researchers have introduced EmoteGPT, a novel framework for generating 3D human facial expressions from natural language descriptions. This system utilizes a multimodal large language model with a specialized token to translate textual inputs into parameters for a 3D Morphable Model. The framework was trained on a new benchmark dataset called Txt2Emote, which includes detailed textual annotations for expressions, and further enhanced with image-to-3DMM data. EmoteGPT demonstrates superior performance in expressiveness and emotion recognition compared to existing text-to-3D face synthesis methods. AI

IMPACT Enables more realistic and controllable 3D avatars and virtual interactions through advanced text-to-expression generation.

RANK_REASON The cluster contains a research paper describing a new model and dataset. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• 3D Morphable Model
• arXiv
• DagsHub
• EmoteGPT
• GPT-4o
• Hugging Face
• multimodal large language model
• Txt2Emote

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →