Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction
Rui Peng,Shihe Shen,Kaiqiang Xiong,Huachen Gao,Jianbo Jiao,Xiaodong Gu,Ronggang Wang
2024-09-05
Abstract:Reconstructing the high-fidelity surface from multi-view images, especially sparse images, is a critical and practical task that has attracted widespread attention in recent years. However, existing methods are impeded by the memory constraint or the requirement of ground-truth depths and cannot recover satisfactory geometric details. To this end, we propose SuRF, a new Surface-centric framework that incorporates a new Region sparsification based on a matching Field, achieving good trade-offs between performance, efficiency and scalability. To our knowledge, this is the first unsupervised method achieving end-to-end sparsification powered by the introduced matching field, which leverages the weight distribution to efficiently locate the boundary regions containing surface. Instead of predicting an SDF value for each voxel, we present a new region sparsification approach to sparse the volume by judging whether the voxel is inside the surface region. In this way, our model can exploit higher frequency features around the surface with less memory and computational consumption. Extensive experiments on multiple benchmarks containing complex large-scale scenes show that our reconstructions exhibit high-quality details and achieve new state-of-the-art performance, i.e., 46% improvements with 80% less memory consumption. Code is available at <a class="link-external link-https" href="https://github.com/prstrive/SuRF" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition