Camera, LiDAR, and IMU Based Multi-Sensor Fusion SLAM: A Survey

Jun Zhu,Hongyi Li,Tao Zhang

DOI: https://doi.org/10.26599/tst.2023.9010010

2024-04-01

Abstract:In recent years, Simultaneous Localization And Mapping (SLAM) technology has prevailed in a wide range of applications, such as autonomous driving, intelligent robots, Augmented Reality (AR), and Virtual Reality (VR). Multi-sensor fusion using the most popular three types of sensors (e.g., visual sensor, LiDAR sensor, and IMU) is becoming ubiquitous in SLAM, in part because of the complementary sensing capabilities and the inevitable shortages (e.g., low precision and long-term drift) of the stand-alone sensor in challenging environments. In this article, we survey thoroughly the research efforts taken in this field and strive to provide a concise but complete review of the related work. Firstly, a brief introduction of the state estimator formation in SLAM is presented. Secondly, the state-of-the-art algorithms of different multi-sensor fusion algorithms are given. Then we analyze the deficiencies associated with the reviewed approaches and formulate some future research considerations. This paper can be considered as a brief guide to newcomers and a comprehensive reference for experienced researchers and engineers to explore new interesting orientations.

computer science, information systems,engineering, electrical & electronic, software engineering

What problem does this paper attempt to address?

The paper mainly addresses several key issues in multi-sensor fusion SLAM (Simultaneous Localization And Mapping) technology, specifically including the following aspects:

1. **Complementary Sensing Capabilities**: The performance of a single sensor (such as a visual sensor, LiDAR, or IMU) is limited in complex environments. For example, visual sensors face scale drift issues in textureless environments, LiDAR fails in poorly structured environments, and IMU accumulates errors during long-term navigation. By fusing multiple sensors, these individual shortcomings can be complemented, improving the overall accuracy of localization and mapping.

2. **Overview of Sensor Fusion Algorithms**: The paper provides a comprehensive review of four types of sensor fusion algorithms: visual-inertial, LiDAR-inertial, visual-LiDAR, and LiDAR-visual-inertial. It focuses on how to compensate for the deficiencies of different sensors to achieve more reliable position estimation.

3. **Introduction to State Estimators**: The paper briefly introduces two commonly used state estimation methods in multi-sensor fusion—Kalman Filter (KF) and Sliding Window Optimization (SWO). It also describes in detail techniques such as Extended Kalman Filter (EKF), Iterated Extended Kalman Filter (IEKF), and Error-State Kalman Filter (ESKF).

4. **Future Research Directions**: Finally, the paper discusses the challenges currently faced by multi-sensor fusion SLAM technology and points out future research directions. The aim is to provide a guide for beginners and also offer references for experienced researchers and engineers to explore new and interesting directions.