New ViDS system uses 3D face tracking for realistic portrait animation

Researchers have developed ViDS, a novel Video Diffusion Shader that utilizes 3D face tracking to create realistic and identity-preserving portrait animations. The system reconstructs a 3D Morphable Model (3DMM) mesh from a reference image and animates it using expression and pose data from a driving video. ViDS employs a video diffusion model as a neural shader, leveraging dense geometric cues from 3DMM normal maps to generate lifelike animations that maintain the original appearance and identity. The method demonstrates improved control over expression and pose compared to previous approaches, with an autoregressive diffusion sampling process enhancing consistency across generated clips. AI

IMPACT This research could lead to more sophisticated tools for creating realistic digital avatars and enhancing video content with personalized animations.

RANK_REASON The cluster describes a new research paper detailing a novel method for portrait animation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• 3D Morphable Model
• arXiv
• neural shader
• Portrait Animation
• Video Diffusion Model

• paper
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →