GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering

Yanyan Li,Chenyu Lyu,Yan Di,Guangyao Zhai,Gim Hee Lee,Federico Tombari

2024-07-17

Abstract:During the Gaussian Splatting optimization process, the scene's geometry can gradually deteriorate if its structure is not deliberately preserved, especially in non-textured regions such as walls, ceilings, and furniture surfaces. This degradation significantly affects the rendering quality of novel views that deviate significantly from the viewpoints in the training data. To mitigate this issue, we propose a novel approach called GeoGaussian. Based on the smoothly connected areas observed from point clouds, this method introduces a novel pipeline to initialize thin Gaussians aligned with the surfaces, where the characteristic can be transferred to new generations through a carefully designed densification strategy. Finally, the pipeline ensures that the scene's geometry and texture are maintained through constrained optimization processes with explicit geometry constraints. Benefiting from the proposed architecture, the generative ability of 3D Gaussians is enhanced, especially in structured regions. Our proposed pipeline achieves state-of-the-art performance in novel view synthesis and geometric reconstruction, as evaluated qualitatively and quantitatively on public datasets.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper proposes a solution to the problem of geometric information loss in 3D high-quality rendering, particularly for non-textured areas such as walls, ceilings, and furniture surfaces. In the traditional Gaussian projection optimization process, the scene's geometric structure may gradually degrade, resulting in a decrease in rendering quality for new views with significant differences from the training data perspectives. To address this issue, the paper introduces the GeoGaussian method, which initializes a thin Gaussian distribution aligned with the surface based on the observed smooth connectivity regions in the point cloud and transfers features to the new generation of Gaussian distributions through a carefully designed densification strategy. The constraint optimization process ensures the preservation of scene geometry and texture, thereby enhancing the generation capability of 3D Gaussian distributions, especially in structured regions. Experimental results demonstrate that GeoGaussian achieves state-of-the-art performance in novel view synthesis and geometric reconstruction.