Glove2Hand framework synthesizes realistic bare-hand interactions from sensor gloves

Researchers have developed Glove2Hand, a new framework that translates data from multi-modal sensing gloves into photorealistic videos of bare hands interacting with objects. This method preserves crucial physical dynamics like contact forces and motion, which are often missing in traditional video recordings. The system uses a novel 3D Gaussian hand model for temporal consistency and a diffusion-based restorer for seamless scene integration, effectively handling occlusions and deformations. The framework also introduces HandSense, a new dataset for hand-object interaction (HOI) that synchronizes glove data with video, aiming to improve applications like contact estimation and hand tracking. AI

IMPACT Enhances realism and physical accuracy in virtual and robotic hand interactions, improving applications like AR/VR and robotics.

RANK_REASON The cluster contains a research paper detailing a new framework and dataset for synthesizing hand-object interactions. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Glove2Hand
• Xinyu Zhang

• paper
• product
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →