Researchers have developed PAGE, a novel framework designed to improve the generation of realistic, audio-driven portrait videos that synchronize with speech and gestures. This system enhances local details and audio-visual alignment by using Parts-Specific Reweighting during training to focus on critical regions like lips and eyes, and employing Consistency Classifier Guidance and Differential Guidance during inference to correct inconsistencies and refine motion. To support evaluation, a new benchmark dataset called Chinese Announcer Speech (CAS) has been created, featuring greater diversity than existing datasets. AI
IMPACT Enhances realism and synchronization in AI-generated videos, potentially impacting applications in virtual avatars and content creation.
RANK_REASON The cluster describes a new research paper detailing a novel framework and dataset for video generation. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- Chinese Announcer Speech
- Consistency Classifier Guidance
- DagsHub
- Differential Guidance
- Hugging Face
- PAGE
- Parts-Specific Reweighting
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
