GauStudio: A Modular Framework for 3D Gaussian Splatting and Beyond

Chongjie Ye,Yinyu Nie,Jiahao Chang,Yuantao Chen,Yihao Zhi,Xiaoguang Han

2024-03-29

Abstract:We present GauStudio, a novel modular framework for modeling 3D Gaussian Splatting (3DGS) to provide standardized, plug-and-play components for users to easily customize and implement a 3DGS pipeline. Supported by our framework, we propose a hybrid Gaussian representation with foreground and skyball background models. Experiments demonstrate this representation reduces artifacts in unbounded outdoor scenes and improves novel view synthesis. Finally, we propose Gaussian Splatting Surface Reconstruction (GauS), a novel render-then-fuse approach for high-fidelity mesh reconstruction from 3DGS inputs without fine-tuning. Overall, our GauStudio framework, hybrid representation, and GauS approach enhance 3DGS modeling and rendering capabilities, enabling higher-quality novel view synthesis and surface reconstruction.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper proposes GauStudio, a modular framework for 3D Gaussian scattering, aiming to address issues in 3D scene modeling and novel view synthesis. GauStudio supports flexible customization through standardized modules to adapt to various task requirements such as scene reconstruction, editing, and simulation. It improves the 3D Gaussian representation and reduces artifacts in outdoor scenes, and introduces a new rendering fusion method called GauS for generating high-fidelity meshes from Gaussian scattering data without fine-tuning. Additionally, GauStudio also supports background modeling, such as sky background modeling, to enhance the realism and quality of the scenes.