F-3DGS: Factorized Coordinates and Representations for 3D Gaussian Splatting

Xiangyu Sun,Joo Chan Lee,Daniel Rho,Jong Hwan Ko,Usman Ali,Eunbyung Park

2024-05-28

Abstract:The neural radiance field (NeRF) has made significant strides in representing 3D scenes and synthesizing novel views. Despite its advancements, the high computational costs of NeRF have posed challenges for its deployment in resource-constrained environments and real-time applications. As an alternative to NeRF-like neural rendering methods, 3D Gaussian Splatting (3DGS) offers rapid rendering speeds while maintaining excellent image quality. However, as it represents objects and scenes using a myriad of Gaussians, it requires substantial storage to achieve high-quality representation. To mitigate the storage overhead, we propose Factorized 3D Gaussian Splatting (F-3DGS), a novel approach that drastically reduces storage requirements while preserving image quality. Inspired by classical matrix and tensor factorization techniques, our method represents and approximates dense clusters of Gaussians with significantly fewer Gaussians through efficient factorization. We aim to efficiently represent dense 3D Gaussians by approximating them with a limited amount of information for each axis and their combinations. This method allows us to encode a substantially large number of Gaussians along with their essential attributes -- such as color, scale, and rotation -- necessary for rendering using a relatively small number of elements. Extensive experimental results demonstrate that F-3DGS achieves a significant reduction in storage costs while maintaining comparable quality in rendered images.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the issues present in the 3D Gaussian Splatting (3DGS) method, particularly its requirement for a large amount of storage space and the resulting high memory demand when representing high-quality images. As a fast rendering technique, 3DGS can achieve high-speed rendering while maintaining excellent image quality. However, it relies on a large number of 3D Gaussian elements to sustain high-fidelity image representation, leading to increased storage costs.

To solve the aforementioned problems, the paper proposes a new method called Factorized 3D Gaussian Splatting (F-3DGS). This method reduces storage overhead while maintaining image quality through the following approaches:
1. **Coordinate Factorization**: Inspired by classical matrix and tensor decomposition techniques, F-3DGS adopts a new coordinate representation method that can efficiently approximate dense clusters of 3D Gaussian elements with a significantly smaller number of Gaussians, thereby greatly reducing storage requirements.
2. **Attribute Factorization**: In addition to coordinates, F-3DGS also factorizes the attributes associated with each Gaussian element (such as color, scale, rotation, etc.), further compressing the model size while preserving the key features of each Gaussian element.
3. **Binary Masking**: To further enhance efficiency, the study introduces binary masking to remove unnecessary Gaussian elements, which helps accelerate the training and rendering process with almost no additional storage overhead.

Through these improvements, F-3DGS can significantly reduce storage costs while maintaining image quality comparable to the original 3DGS. Experimental results show that F-3DGS can achieve image quality close to or even better than existing methods while significantly reducing model size and providing faster rendering speeds.