New Vision Transformer framework enhances sparse dToF depth completion

Researchers have developed a new framework for dense metric depth completion from sparse direct Time-of-Flight (dToF) sensors. This method utilizes a depth-guided dual-branch Vision Transformer encoder that processes RGB images and sparse dToF measurements separately, with a masked joint attention module enabling depth tokens to guide image features effectively. The system is trained entirely on synthetic data generated by a comprehensive dToF simulation pipeline, which replicates various sensor types and degradation characteristics. This approach achieves strong zero-shot generalization across multiple datasets and real-world dToF devices, outperforming existing methods in accuracy and computational efficiency. AI

IMPACT This research could improve 3D perception for applications like robotics and extended reality by enabling denser depth maps from sparse sensor data.

RANK_REASON The cluster contains a research paper detailing a new method for depth completion from sensor data. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• dToF
• extended reality
• RGB color model
• time of flight
• virtual reality
• vision transformer

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →