Abstract:Reconstruction of a continuous surface of two-dimensional manifold from its raw, discrete point cloud observation is a long-standing problem in computer vision and graphics research. The problem is technically ill-posed, and becomes more difficult considering that various sensing imperfections would appear in the point clouds obtained by practical depth scanning. In literature, a rich set of methods has been proposed, and reviews of existing methods are also provided. However, existing reviews are short of thorough investigations on a common benchmark. The present paper aims to review and benchmark existing methods in the new era of deep learning surface reconstruction. To this end, we contribute a large-scale benchmarking dataset consisting of both synthetic and real-scanned data; the benchmark includes object- and scene-level surfaces and takes into account various sensing imperfections that are commonly encountered in practical depth scanning. We conduct thorough empirical studies by comparing existing methods on the constructed benchmark, and pay special attention on robustness of existing methods against various scanning imperfections; we also study how different methods generalize in terms of reconstructing complex surface shapes. Our studies help identity the best conditions under which different methods work, and suggest some empirical findings. For example, while deep learning methods are increasingly popular in the research community, our systematic studies suggest that, surprisingly, a few classical methods perform even better in terms of both robustness and generalization; our studies also suggest that the practical challenges of misalignment of point sets from multi-view scanning, missing of surface points, and point outliers remain unsolved by all the existing surface reconstruction methods. We expect that the benchmark and our studies would be valuable both for practitioners and as a guidance for new innovations in future research. We make the benchmark publicly accessible at https://Gorilla-Lab-SCUT.github.io/SurfaceReconstructionBenchmark.

Benchmark datasets for 3D computer vision

Benchmarking Large-Scale Multi-View 3D Reconstruction Using Realistic Synthetic Images

Monocular Image-Based 3-D Model Retrieval: A Benchmark

BelHouse3D: A Benchmark Dataset for Assessing Occlusion Robustness in 3D Point Cloud Semantic Segmentation

SUN RGB-D: A RGB-D scene understanding benchmark suite

UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles

A Benchmark Grocery Dataset of Realworld Point Clouds From Single View

3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

View-Based 3-D Model Retrieval: A Benchmark

An Aerial Photogrammetry Benchmark Dataset for Point Cloud Segmentation and Style Translation

Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds

SS3DM: Benchmarking Street-View Surface Reconstruction with a Synthetic 3D Mesh Dataset

Surface Reconstruction from Point Clouds: A Survey and a Benchmark

VEnvision3D: A Synthetic Perception Dataset for 3D Multi-Task Model Research

2D and 3D Object Detection Algorithms from Images: A Survey

Survey on Vision-Based 3D Object Detection Methods

Deep Learning Based 3D Segmentation: A Survey

3DBench: A Scalable 3D Benchmark and Instruction-Tuning Dataset

GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark