Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving
Researchers have developed Sensor2Sensor, a new generative modeling approach to convert in-the-wild dashcam videos into structured, multi-modal sensor data suitable for autonomous driving systems. This method addresses the challenge of limited proprietary datasets by leveraging the vast scale and diversity of publicly available video footage. Sensor2Sensor utilizes a diffusion architecture and 4D Gaussian Splatting to generate realistic multi-view camera images and LiDAR point clouds from monocular videos, thereby unlocking new data sources for AV development. AI
IMPACT Enables the use of vast public video datasets for training and validating autonomous driving systems, potentially accelerating development.