How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey

Fabio Tosi,Youmin Zhang,Ziren Gong,Erik Sandström,Stefano Mattoccia,Martin R. Oswald,Matteo Poggi

2024-04-11

Abstract:Over the past two decades, research in the field of Simultaneous Localization and Mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evolution ranges from hand-crafted methods, through the era of deep learning, to more recent developments focused on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) representations. Recognizing the growing body of research and the absence of a comprehensive survey on the topic, this paper aims to provide the first comprehensive overview of SLAM progress through the lens of the latest advancements in radiance fields. It sheds light on the background, evolutionary path, inherent strengths and limitations, and serves as a fundamental reference to highlight the dynamic progress and specific challenges.

Robotics

What problem does this paper attempt to address?

The paper primarily explores how Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have reshaped the development direction of technology in the field of Simultaneous Localization and Mapping (SLAM) and provides a comprehensive review of these advancements.

The paper points out that over the past 20 years, SLAM technology has undergone significant evolution, from manually designed methods to the era of deep learning, and more recently to radiance field representation methods represented by NeRF and 3DGS. The core goal of SLAM technology is to enable machines to autonomously navigate in unknown environments while simultaneously constructing a map of the environment and determining their own pose. With advancements in computer vision, robotics, and sensor technology, the application range of SLAM technology has continuously expanded, including fields such as Augmented Reality (AR), visual surveillance, medical applications, and more.

However, traditional SLAM methods perform poorly when faced with strong lighting changes, dynamic or texture-poor environments. In recent years, the introduction of deep learning technology has improved the accuracy and reliability of SLAM systems, but there still exists the problem of relying on large amounts of training data, making it difficult to generalize well to unseen scenes. Additionally, methods based on discrete surface representations (such as point clouds, voxel grids, etc.) have limitations in terms of sparsity in 3D modeling and spatial resolution.

To overcome these challenges, radiance field representation methods such as NeRF and 3DGS have been introduced into the SLAM field. These methods bring revolutionary changes to SLAM technology through continuous surface modeling, reduced memory requirements, improved noise handling capabilities, and effective scene filling in occluded or sparsely observed areas. Although these new technologies are still in their early stages, they have already shown great potential in addressing the shortcomings of existing technologies.

This paper aims to fill the current gap in the review of the latest advancements in SLAM technology by providing an in-depth analysis of 73 SLAM systems that have emerged in the past 3 years, offering readers a comprehensive understanding perspective. The paper not only reviews relevant background knowledge and technical principles but also details the characteristics of NeRF and 3DGS technologies and looks forward to future research directions.