New framework adapts 3D reconstruction models for fisheye lenses

Researchers have developed Fisheye3R, a new framework designed to adapt existing 3D reconstruction foundation models to effectively process images from fisheye lenses. These models, typically trained on standard perspective images, suffer performance degradation when encountering the high radial distortion of fisheye lenses. Fisheye3R enables these models to handle fisheye inputs without compromising performance on perspective images, even with limited fisheye training data. Experiments show improvements in camera pose, depth, and point map estimation across multiple foundation models like VGGT-Ω, $\pi^3$, and MapAnything. AI

IMPACT Enables more robust 3D reconstruction from a wider variety of camera inputs, potentially improving applications in robotics and autonomous systems.

RANK_REASON Academic paper detailing a new method for adapting existing models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• Fisheye3R
• MapAnything
• $\\pi^3$
• Ruxiao Duan
• VGGT-Ω

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →