UniSwap framework enables real-time audio-visual identity swapping in videos

Researchers have introduced UniSwap, a novel framework for real-time audio-visual identity swapping in talking videos. This system integrates appearance and voice transfer within a single diffusion transformer, aiming to improve audio-visual consistency compared to methods that handle modalities separately. UniSwap addresses the challenge of limited aligned training data by employing a swap-and-reconstruct pipeline and introduces several adaptations for efficient streaming and stable long-form generation. AI

IMPACT This research could lead to more seamless and efficient video editing tools for content creation and digital avatars.

RANK_REASON The cluster contains a research paper detailing a new technical framework for audio-visual identity swapping. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• ScienceCast
• UniSwap

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →