DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus

Yu Chen,Gim Hee Lee

2024-10-29

Abstract:The recent advances in 3D Gaussian Splatting (3DGS) show promising results on the novel view synthesis (NVS) task. With its superior rendering performance and high-fidelity rendering quality, 3DGS is excelling at its previous NeRF counterparts. The most recent 3DGS method focuses either on improving the instability of rendering efficiency or reducing the model size. On the other hand, the training efficiency of 3DGS on large-scale scenes has not gained much attention. In this work, we propose DoGaussian, a method that trains 3DGS distributedly. Our method first decomposes a scene into K blocks and then introduces the Alternating Direction Method of Multipliers (ADMM) into the training procedure of 3DGS. During training, our DOGS maintains one global 3DGS model on the master node and K local 3DGS models on the slave nodes. The K local 3DGS models are dropped after training and we only query the global 3DGS model during inference. The training time is reduced by scene decomposition, and the training convergence and stability are guaranteed through the consensus on the shared 3D Gaussians. Our method accelerates the training of 3DGS by 6+ times when evaluated on large-scale scenes while concurrently achieving state-of-the-art rendering quality. Our code is publicly available at <a class="link-external link-https" href="https://github.com/AIBluefisher/DOGS" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problems that this paper attempts to solve are two main challenges encountered when training 3D Gaussian Splatting (3DGS) models in large - scale scenarios: 1) high GPU memory requirements to accommodate large - scale 3D models; 2) long - term training due to the large area of the scene. Existing large - scale NeRF methods solve these problems by adopting a divide - and - conquer approach, that is, dividing the scene into multiple small blocks and fitting each small block with a smaller model. However, these methods need to query multiple sub - models during inference, which will reduce the rendering efficiency.

To solve the above problems, the paper proposes DOGS (Distributed - Oriented Gaussian Splatting), a method for distributed training of 3DGS. DOGS decomposes the scene into multiple blocks and introduces the Alternating Direction Method of Multipliers (ADMM) to ensure the consistency of shared 3D Gaussians between different blocks, thereby reducing the training time and ensuring the convergence and stability of training. Specifically, DOGS maintains a global 3DGS model on the master node and multiple local 3DGS models on the slave nodes. During the training process, the global 3DGS model is updated by collecting and averaging local 3D Gaussians, and the updated global 3DGS model is shared with all slave nodes at the end of each training iteration to regulate the training of local 3D Gaussians. In this way, DOGS not only accelerates the training of 3DGS but also achieves state - of - the - art rendering quality in large - scale scenarios.

The main contributions of the paper include:
- Proposing a recursive method to divide the scene into evenly - sized blocks.
- Introducing a distributed method for 3D Gaussian training in large - scale scenarios, which reduces the training time by more than six times compared with the original 3DGS.
- Conducting exhaustive experiments on standard large - scale datasets to verify the effectiveness of the method.