Abstract:3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in scene synthesis and novel view synthesis tasks. Typically, the initialization of 3D Gaussian primitives relies on point clouds derived from Structure-from-Motion (SfM) methods. However, in scenarios requiring scene reconstruction from sparse viewpoints, the effectiveness of 3DGS is significantly constrained by the quality of these initial point clouds and the limited number of input images. In this study, we present Dust-GS, a novel framework specifically designed to overcome the limitations of 3DGS in sparse viewpoint conditions. Instead of relying solely on SfM, Dust-GS introduces an innovative point cloud initialization technique that remains effective even with sparse input data. Our approach leverages a hybrid strategy that integrates an adaptive depth-based masking technique, thereby enhancing the accuracy and detail of reconstructed scenes. Extensive experiments conducted on several benchmark datasets demonstrate that Dust-GS surpasses traditional 3DGS methods in scenarios with sparse viewpoints, achieving superior scene reconstruction quality with a reduced number of input images.
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve
This paper aims to address the shortcomings of the 3D Gaussian Splatting (3DGS) method in terms of point cloud quality and quantity when reconstructing scenes from sparse views (i.e., a limited number of input images). Specifically, traditional 3DGS methods rely on initial point clouds generated from Structure-from-Motion (SfM) methods, but under sparse view conditions, these point clouds often lack sufficient detail and accuracy, thereby limiting the performance of 3DGS.
### Main Contributions
1. **Proposed a new framework**: Dust-GS, which optimizes Gaussian Splatting to enable high-quality 3D scene reconstruction under sparse input data.
2. **Introduced a new point cloud initialization strategy**: This reduces reliance on traditional SfM methods, generating denser and more accurate point clouds, thereby improving the quality of scene synthesis.
3. **Developed a dynamic depth mask mechanism**: By controlling Gaussian primitives, it effectively filters noise, enhances geometric fidelity, and improves the quality of the reconstructed scene.
### Experimental Results
Experimental results show that Dust-GS outperforms other methods on multiple benchmark datasets (such as MipNeRF360 and BungeeNeRF), especially in handling complex scenes, better recovering details, and maintaining geometric consistency. Specifically, it excels in the following aspects:
- **Quantitative evaluation**: Dust-GS outperforms other methods in metrics such as PSNR, SSIM, and LPIPS.
- **Qualitative analysis**: Visual effects are significantly improved, especially in scenes with reflective objects and complex lighting conditions, where Dust-GS reduces blur and distortion, achieving higher reconstruction accuracy.
### Conclusion
By introducing a dynamic depth mask and a new point cloud initialization strategy, Dust-GS effectively addresses the performance degradation of 3DGS under sparse view conditions, enhancing the accuracy and robustness of scene reconstruction. This method has broad application prospects in fields such as robotics, autonomous driving, and virtual reality.