SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction

Jialei Chen,Xin Zhang,Mobarakol Islam,Francisco Vasconcelos,Danail Stoyanov,Daniel S. Elson,Baoru Huang

2024-10-12

Abstract:Accurate 3D reconstruction of dynamic surgical scenes from endoscopic video is essential for robotic-assisted surgery. While recent 3D Gaussian Splatting methods have shown promise in achieving high-quality reconstructions with fast rendering speeds, their use of inverse depth loss functions compresses depth variations. This can lead to a loss of fine geometric details, limiting their ability to capture precise 3D geometry and effectiveness in intraoperative application. To address these challenges, we present SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction with improved geometric accuracy. Our approach first initialises a Gaussian point cloud using depth priors, employing binary motion masks to identify pixels with significant depth variations and fusing point clouds from depth maps across frames for initialisation. We use the Flexible Deformation Model to represent dynamic scene and introduce a normalised depth regularisation loss along with an unsupervised depth smoothness constraint to ensure more accurate geometric reconstruction. Extensive experiments on two real surgical datasets demonstrate that SurgicalGS achieves state-of-the-art reconstruction quality, especially in terms of accurate geometry, advancing the usability of 3D Gaussian Splatting in robotic-assisted surgery.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the problem of accurately reconstructing the 3D structure of dynamic surgical scenes from endoscopic videos in robot-assisted surgery. Specifically, while existing 3D Gaussian Splatting methods show potential in achieving high-quality reconstruction and fast rendering speeds, their use of inverse depth loss functions compresses depth variations, leading to the loss of fine geometric details and limiting their ability to accurately reconstruct 3D geometry in intraoperative applications. To this end, the paper proposes SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction, aiming to improve geometric accuracy.

The main contributions of the paper include:
1. **Dense Initialization**: Utilizing geometric information from all frames for dense Gaussian point initialization to enhance reconstruction quality.
2. **Normalized Depth Regularization and Unsupervised Depth Smoothing**: Introducing a normalized depth regularization loss and unsupervised depth smoothing constraints to better leverage depth prior information, thereby improving the accuracy of geometric reconstruction.
3. **Experimental Validation**: Extensive experiments on two public datasets demonstrate the superior performance of SurgicalGS in 3D surgical scene reconstruction, particularly in terms of geometric accuracy.

These improvements make SurgicalGS more reliable and efficient for applications in robot-assisted surgery.