Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves
Researchers have developed Glove2Hand, a new framework that translates data from multi-modal sensing gloves into photorealistic videos of bare hands interacting with objects. This method preserves crucial physical dynamics like contact forces and motion, which are often missing in traditional video recordings. The system uses a novel 3D Gaussian hand model for temporal consistency and a diffusion-based restorer for seamless scene integration, effectively handling occlusions and deformations. The framework also introduces HandSense, a new dataset for hand-object interaction (HOI) that synchronizes glove data with video, aiming to improve applications like contact estimation and hand tracking. AI
IMPACT Enhances realism and physical accuracy in virtual and robotic hand interactions, improving applications like AR/VR and robotics.