Abstract:Recently, the multi-modal fusion of RGB, depth, and semantics has shown great potential in the domain of dense Simultaneous Localization and Mapping (SLAM), as known as dense semantic SLAM. Yet a prerequisite for generating consistent and continuous semantic maps is the availability of dense, efficient, and scalable scene representations. To date, existing semantic SLAM systems based on explicit scene representations (points/meshes/surfels) are limited by their resolutions and inabilities to predict unknown areas, thus failing to generate dense maps. Contrarily, a few implicit scene representations (Neural Radiance Fields) to deal with these problems rely on time-consuming ray tracing-based volume rendering technique, which cannot meet the real-time rendering requirements of SLAM. Fortunately, the Gaussian Splatting scene representation has recently emerged, which inherits the efficiency and scalability of point/surfel representations while smoothly represents geometric structures in a continuous manner, showing promise in addressing the aforementioned challenges. To this end, we propose GS3LAM, a Gaussian Semantic Splatting SLAM framework, which takes multimodal data as input and can render consistent, continuous dense semantic maps in real-time. To fuse multimodal data, GS3LAM models the scene as a Semantic Gaussian Field (SG-Field), and jointly optimizes camera poses and the field by establishing error constraints between observed and predicted data. Furthermore, a Depth-adaptive Scale Regularization (DSR) scheme is proposed to tackle the problem of misalignment between scale-invariant Gaussians and geometric surfaces within the SG-Field. To mitigate the forgetting phenomenon, we propose an effective Random Sampling-based Keyframe Mapping (RSKM) strategy, which exhibits notable superiority over local covisibility optimization strategies commonly utilized in 3DGS-based SLAM systems. Extensive experiments conducted on the benchmark datasets reveal that compared with state-of-the-art competitors, GS3 LAM demonstrates increased tracking robustness, superior real-time rendering quality, and enhanced semantic reconstruction precision. To make the results reproducible, the source code is available at https://github.com/lif314/GS3LAM.

Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mapping

Real-time Semantic 3D Dense Occupancy Mapping with Efficient Free Space Representations

Autonomous Navigation in Unknown Environments With Sparse Bayesian Kernel-Based Occupancy Mapping

S-MKI: Incremental Dense Semantic Occupancy Reconstruction Through Multi-Entropy Kernel Inference

Semantic 3D occupancy mapping through efficient high order CRFs

City-scale Continual Neural Semantic Mapping with Three-Layer Sampling and Panoptic Representation

GS3LAM: Gaussian Semantic Splatting SLAM

Multimodal sensor-based semantic 3D mapping for a large-scale environment

SEE-CSOM: Sharp-Edged and Efficient Continuous Semantic Occupancy Mapping for Mobile Robots

A Semantic OctoMap Mapping Method Based on CBAM-PSPNet

Probabilistic Data Association for Semantic SLAM at Scale

Particle-based Instance-aware Semantic Occupancy Mapping in Dynamic Environments

Probabilistic Projective Association and Semantic Guided Relocalization for Dense Reconstruction

Collaborative Probabilistic Semantic Mapping Using CNN

Probabilistic Semantic Mapping for Autonomous Driving in Urban Environments

Evidential Semantic Mapping in Off-road Environments with Uncertainty-aware Bayesian Kernel Inference

Semi-Dense 3D Semantic Mapping from Monocular SLAM

Monocular Semantic Mapping Based on 3D Cuboids Tracking.

Utilization of Semantic Planes: Improved Localization and Dense Semantic Map for Monocular SLAM in Urban Environment

Large-scale 3D Semantic Mapping Using Stereo Vision

Robust Dense Mapping for Large-Scale Dynamic Environments