Can These Views Be One Scene? Evaluating Multiview 3D Consistency when 3D Foundation Models Hallucinate
Researchers have developed a new benchmark, \benchmark, to evaluate the consistency of 3D reconstructions from multiple camera views, particularly when 3D foundation models hallucinate details. This benchmark compares neural reconstruction priors with classical geometric verification methods. The study found that existing metrics like MEt3R can incorrectly assign high scores to inconsistent or artifact-laden outputs, while the new COLMAP-based metrics show a significantly higher correlation with human judgments. AI
IMPACT Introduces a new evaluation framework to better assess the reliability of 3D foundation models, crucial for applications in computer vision and generative AI.