DebSDF: Delving into the Details and Bias of Neural Indoor Scene Reconstruction

Yuting Xiao,Jingwei Xu,Zehao Yu,Shenghua Gao

2024-07-11

Abstract:In recent years, the neural implicit surface has emerged as a powerful representation for multi-view surface reconstruction due to its simplicity and state-of-the-art performance. However, reconstructing smooth and detailed surfaces in indoor scenes from multi-view images presents unique challenges. Indoor scenes typically contain large texture-less regions, making the photometric loss unreliable for optimizing the implicit surface. Previous work utilizes monocular geometry priors to improve the reconstruction in indoor scenes. However, monocular priors often contain substantial errors in thin structure regions due to domain gaps and the inherent inconsistencies when derived independently from different views. This paper presents \textbf{DebSDF} to address these challenges, focusing on the utilization of uncertainty in monocular priors and the bias in SDF-based volume rendering. We propose an uncertainty modeling technique that associates larger uncertainties with larger errors in the monocular priors. High-uncertainty priors are then excluded from optimization to prevent bias. This uncertainty measure also informs an importance-guided ray sampling and adaptive smoothness regularization, enhancing the learning of fine structures. We further introduce a bias-aware signed distance function to density transformation that takes into account the curvature and the angle between the view direction and the SDF normals to reconstruct fine details better. Our approach has been validated through extensive experiments on several challenging datasets, demonstrating improved qualitative and quantitative results in reconstructing thin structures in indoor scenes, thereby outperforming previous work.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to address the challenges encountered in reconstructing indoor scenes from multi - view images, especially how to reconstruct smooth and detailed - rich surfaces from multi - view images. Specifically, the paper points out the following main issues:

1. **Inherent Uncertainty in Multi - view Reconstruction**: Multi - view reconstruction is an under - constrained problem because there are countless possible implicit surfaces that can match the input images. This complicates the optimization process.

2. **Problems in Texture - less Areas**: Indoor scenes usually contain large areas of texture - less regions, making the photometric - loss - based optimization unreliable. These areas lack sufficient visual information, causing the reconstruction results to be error - prone.

3. **Errors in Monocular Geometric Priors**: Although monocular geometric priors (such as monocular depth and normal maps) can improve the reconstruction quality, in fine - structure areas (such as chair legs and table feet), these priors often have large errors. These errors may be caused by the domain gap between the training data and the actual scene and the inconsistency of independent predictions from different viewpoints.

4. **Low Sampling Probability in Fine - structure Areas**: Fine - structure areas occupy a small area in the input images. Existing methods usually sample all training images uniformly, resulting in a low sampling probability in these areas, which affects the learning of fine structures.

5. **Impact of Smoothing Regularization**: Existing methods apply a uniform smoothing regularization to the entire space, which will inhibit the learning of fine - structure surfaces.

To solve these problems, the paper proposes the **DebSDF** method, and its main contributions include:

- **Uncertainty Modeling**: Through uncertainty modeling techniques, larger uncertainties are associated with larger prior errors, and priors with high uncertainty are excluded from the optimization to reduce bias.

- **Importance - guided Ray Sampling**: Use the uncertainty map to guide ray sampling, increase the sampling probability of fine - structure areas, and thus improve the learning effect of fine structures.

- **Adaptive Smoothing Regularization**: Apply a smaller smoothing regularization in fine - structure areas to promote the learning of fine structures.

- **Bias - aware SDF - to - Density Conversion**: Propose an SDF - to - density conversion method that takes into account the curvature and the angle between the view direction and the SDF normal, reducing the bias in volume rendering and better reconstructing fine structures.

Through these methods, DebSDF has been verified on multiple challenging datasets, showing significant improvements in reconstructing the fine structures of indoor scenes.