G2SDF: Surface Reconstruction from Explicit Gaussians with Implicit SDFs

Kunyi Li,Michael Niemeyer,Zeyu Chen,Nassir Navab,Federico Tombari

2024-11-26

Abstract:State-of-the-art novel view synthesis methods such as 3D Gaussian Splatting (3DGS) achieve remarkable visual quality. While 3DGS and its variants can be rendered efficiently using rasterization, many tasks require access to the underlying 3D surface, which remains challenging to extract due to the sparse and explicit nature of this representation. In this paper, we introduce G2SDF, a novel approach that addresses this limitation by integrating a neural implicit Signed Distance Field (SDF) into the Gaussian Splatting framework. Our method links the opacity values of Gaussians with their distances to the surface, ensuring a closer alignment of Gaussians with the scene surface. To extend this approach to unbounded scenes at varying scales, we propose a normalization function that maps any range to a fixed interval. To further enhance reconstruction quality, we leverage an off-the-shelf depth estimator as pseudo ground truth during Gaussian Splatting optimization. By establishing a differentiable connection between the explicit Gaussians and the implicit SDF, our approach enables high-quality surface reconstruction and rendering. Experimental results on several real-world datasets demonstrate that G2SDF achieves superior reconstruction quality than prior works while maintaining the efficiency of 3DGS.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is that when 3D Gaussian Splatting (3DGS) is used for high - quality view synthesis, although it can be rendered efficiently, its discrete and unstructured representation makes it difficult to accurately extract 3D surfaces from this representation. Specifically, since the optimization process of 3DGS mainly depends on the supervision of rendering quality, this leads to noisy manifestations of the scene geometry, thereby increasing the difficulty of accurately extracting 3D surfaces through post - processing methods such as Poisson surface reconstruction.

To solve these problems, the paper proposes G2SDF, a new method to improve the reconstruction quality of 3D surfaces by integrating the neural implicit Signed Distance Field (SDF) into the Gaussian Splatting framework. G2SDF ensures that Gaussian points are more closely aligned with the scene surface by correlating the opacity of Gaussian points with their distances to the surface. In addition, in order to adapt to unbounded scenes at different scales, the paper also proposes a normalization function that can map any range to a fixed interval. By establishing a differentiable connection between explicit Gaussian points and implicit SDF, this method can achieve high - quality surface reconstruction and rendering while maintaining the efficiency of 3DGS.

Experimental results show that G2SDF achieves higher reconstruction quality than existing methods on multiple real - world datasets while maintaining the efficiency of 3DGS.