Unified Video-Action Joint Denoising for Dexterous Action and Data Generation
Researchers have introduced Donk, a novel unified video-action denoising model designed for dexterous robotic hands. This model operates by modeling the joint distribution of interaction videos and hand trajectories, allowing it to generate future videos and action policies from various conditions. Notably, Donk can also function as a data engine, generating paired video-action rollouts solely from text prompts, thereby enhancing its utility in both action generation and data synthesis. AI
IMPACT Introduces a unified approach for generating dexterous robot actions and synthetic video data, potentially accelerating robotics research and development.