SuperGaussians: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors

Rui Xu,Wenyue Chen,Jiepeng Wang,Yuan Liu,Peng Wang,Lin Gao,Shiqing Xin,Taku Komura,Xin Li,Wenping Wang

2024-11-28

Abstract:Gaussian Splattings demonstrate impressive results in multi-view reconstruction based on Gaussian explicit representations. However, the current Gaussian primitives only have a single view-dependent color and an opacity to represent the appearance and geometry of the scene, resulting in a non-compact representation. In this paper, we introduce a new method called SuperGaussians that utilizes spatially varying colors and opacity in a single Gaussian primitive to improve its representation ability. We have implemented bilinear interpolation, movable kernels, and even tiny neural networks as spatially varying functions. Quantitative and qualitative experimental results demonstrate that all three functions outperform the baseline, with the best movable kernels achieving superior novel view synthesis performance on multiple datasets, highlighting the strong potential of spatially varying functions.

Computer Vision and Pattern Recognition,Graphics,Multimedia

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in multi - view reconstruction, the existing Gaussian point methods perform poorly and are not compact enough when representing complex scenes. Specifically, current Gaussian point methods (such as 2DGS [11] and 3DGS [16]) use a single color and opacity to represent each Gaussian point. This causes them to require a large number of simple Gaussian points to approximate spatially - varying opacity and texture when dealing with scenes with complex geometric structures and appearances, thus wasting a large amount of Gaussian point resources.

To overcome this problem, the paper introduces a new method - SuperGaussians. This method improves its representational ability by using spatially - varying color and opacity in a single Gaussian point. This method enables a single Gaussian point to better fit the complex textures and geometric structures in the scene, improving the effectiveness and compactness of the representation. By implementing bilinear interpolation, movable kernels, and small neural networks as spatially - varying functions, the paper demonstrates the effectiveness and superiority of these methods on multiple datasets, especially in the new - view synthesis task.