Abstract:Conventional geometry-based SLAM systems lack dense 3D reconstruction capabilities since their data association usually relies on feature correspondences. Additionally, learning-based SLAM systems often fall short in terms of real-time performance and accuracy. Balancing real-time performance with dense 3D reconstruction capabilities is a challenging problem. In this paper, we propose a real-time RGB-D SLAM system that incorporates a novel view synthesis technique, 3D Gaussian Splatting, for 3D scene representation and pose estimation. This technique leverages the real-time rendering performance of 3D Gaussian Splatting with rasterization and allows for differentiable optimization in real time through CUDA implementation. We also enable mesh reconstruction from 3D Gaussians for explicit dense 3D reconstruction. To estimate accurate camera poses, we utilize a rotation-translation decoupled strategy with inverse optimization. This involves iteratively updating both in several iterations through gradient-based optimization. This process includes differentiably rendering RGB, depth, and silhouette maps and updating the camera parameters to minimize a combined loss of photometric loss, depth geometry loss, and visibility loss, given the existing 3D Gaussian map. However, 3D Gaussian Splatting (3DGS) struggles to accurately represent surfaces due to the multi-view inconsistency of 3D Gaussians, which can lead to reduced accuracy in both camera pose estimation and scene reconstruction. To address this, we utilize depth priors as additional regularization to enforce geometric constraints, thereby improving the accuracy of both pose estimation and 3D reconstruction. We also provide extensive experimental results on public benchmark datasets to demonstrate the effectiveness of our proposed methods in terms of pose accuracy, geometric accuracy, and rendering performance.

Camera Pose Estimation Using a 3D Gaussian Splatting Radiance Field

No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images

Look Gauss, No Pose: Novel View Synthesis using Gaussian Splatting without Accurate Pose Initialization

6DGS: 6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model

iComMa: Inverting 3D Gaussian Splatting for Camera Pose Estimation via Comparing and Matching

COLMAP-Free 3D Gaussian Splatting

Gaussian Scenes: Pose-Free Sparse-View Scene Reconstruction using Depth-Enhanced Diffusion Priors

Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion

3D Gaussian Splatting for Real-Time Radiance Field Rendering

IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera

Visual SLAM with 3D Gaussian Primitives and Depth Priors Enabling Novel View Synthesis

GSLoc: Efficient Camera Pose Refinement via 3D Gaussian Splatting

GGRt: Towards Pose-free Generalizable 3D Gaussian Splatting in Real-time

Gaussian Splatting SLAM

Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting

6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting

SparseGS: Real-Time 360° Sparse View Synthesis using Gaussian Splatting

The Potential of Neural Radiance Fields and 3D Gaussian Splatting for 3D Reconstruction from Aerial Imagery

GDGS: Gradient Domain Gaussian Splatting for Sparse Representation of Radiance Fields

Object Gaussian for Monocular 6D Pose Estimation from Sparse Views

GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views