NeuralSRender enhances synthetic video realism for autonomous driving

Researchers have developed NeuralSRender, a novel framework designed to enhance the photorealism of synthetic videos used for training autonomous driving perception systems. This approach leverages a pretrained video diffusion transformer and incorporates geometric and material information from G-buffers. The system is trained in two stages, first on synthetic data and then adapted to real-world data with estimated G-buffers, introducing minimal parameter overhead and enabling training on consumer-grade GPUs. Experiments show that perception models trained on NeuralSRender's translated videos achieve improved performance on benchmarks like nuImages and Cityscapes. AI

IMPACT This research could significantly improve the training data for autonomous driving systems, potentially leading to more robust and safer AI perception models.

RANK_REASON The item describes a new research paper detailing a novel method for sim-to-real rendering. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Cityscapes
• Diffusion Transformer
• G-buffer
• Hugging Face
• LoRA+
• NeuralSRender
• nuImages
• video diffusion transformer

• paper
• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →