RaDe-GS: Rasterizing Depth in Gaussian Splatting

Baowen Zhang,Chuan Fang,Rakesh Shrestha,Yixun Liang,Xiaoxiao Long,Ping Tan

2024-06-24

Abstract:Gaussian Splatting (GS) has proven to be highly effective in novel view synthesis, achieving high-quality and real-time rendering. However, its potential for reconstructing detailed 3D shapes has not been fully explored. Existing methods often suffer from limited shape accuracy due to the discrete and unstructured nature of Gaussian splats, which complicates the shape extraction. While recent techniques like 2D GS have attempted to improve shape reconstruction, they often reformulate the Gaussian primitives in ways that reduce both rendering quality and computational efficiency. To address these problems, our work introduces a rasterized approach to render the depth maps and surface normal maps of general 3D Gaussian splats. Our method not only significantly enhances shape reconstruction accuracy but also maintains the computational efficiency intrinsic to Gaussian Splatting. It achieves a Chamfer distance error comparable to NeuraLangelo on the DTU dataset and maintains similar computational efficiency as the original 3D GS methods. Our method is a significant advancement in Gaussian Splatting and can be directly integrated into existing Gaussian Splatting-based methods.

Graphics,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper focuses on a specific technique in 3D reconstruction called Gaussian Splatting. Existing Gaussian Splatting methods perform well in novel view rendering, but their ability in detailed 3D shape reconstruction has not been fully utilized, and they encounter accuracy issues in shape extraction. In particular, simplifying Gaussian Splatting into planar form helps with surface extraction but leads to a decrease in rendering quality and computational efficiency.

To address these issues, the paper proposes RaDe-GS (Rasterizing Depth in Gaussian Splatting), a novel rasterization method for computing depth and surface normal maps of general Gaussian Splatting. This method not only significantly improves the accuracy of shape reconstruction but also maintains the inherent computational efficiency of Gaussian Splatting. It performs on par with NeuraLangelo in terms of accuracy on the DTU dataset, and has similar training efficiency as the original 3D Gaussian Splatting method.

By analyzing the intersection points between rays and Gaussian splatting bodies, the paper finds a closed-form solution that efficiently computes the depth and normal maps. The authors discovered that these intersection points are coplanar under an approximate affine projection, enabling the calculation of the projection depth for each Gaussian splatting body through rasterization based on the derived plane equations. The final depth map is the median depth considering the opacity of each splat, and the normal map is also computed using a similar approach. This method can generate accurate depth and normal maps while maintaining Gaussian Splatting rendering and optimization efficiency.

In conclusion, the paper's main contribution is a novel rasterization method specifically designed for computing depth and normal maps of Gaussian Splatting, aiming to improve the quality of 3D shape reconstruction while maintaining computational efficiency.