One-Shot Novel View and Pose Human Image Synthesis via 3D Prior Guided Diffusion Model
Researchers have developed a novel diffusion model for synthesizing novel human views and poses from a single image. This approach addresses limitations in existing methods, such as handling complex poses with ambiguous 2D keypoints or inaccuracies in generalizable human NeRFs for occluded parts. The proposed model incorporates 3D human priors, specifically 3D normal maps and color prompts, as conditional inputs to guide the denoising process. This allows for high-quality synthesis, including previously unseen or occluded human body parts, and includes a self-reconstruction-based refinement for enhanced detail. Experimental results on public datasets indicate superior performance and generalization capabilities compared to previous methods. AI
IMPACT This research could advance the capabilities of AI in generating realistic human imagery for applications like virtual try-on, gaming, and content creation.