Abstract:A deep learning-based Visual Inertial SLAM technique is proposed in this paper to ensure accurate autonomous localization of mobile robots in environments with dynamic objects. Addressing the limitations of real-time performance in deep learning algorithms and the poor robustness of pure visual geometry algorithms, this paper presents a deep learning-based Visual Inertial SLAM technique. Firstly, a non-blocking model is designed to extract semantic information from images. Then, a motion probability hierarchy model is proposed to obtain prior motion probabilities of feature points. For image frames without semantic information, a motion probability propagation model is designed to determine the prior motion probabilities of feature points. Furthermore, considering that the output of inertial measurements is unaffected by dynamic objects, this paper integrates inertial measurement information to improve the estimation accuracy of feature point motion probabilities. An adaptive threshold-based motion probability estimation method is proposed, and finally, the positioning accuracy is enhanced by eliminating feature points with excessively high motion probabilities. Experimental results demonstrate that the proposed algorithm achieves accurate localization in dynamic environments while maintaining real-time performance.

What problem does this paper attempt to address?

This paper attempts to address the problem of real-time autonomous localization of mobile robots in dynamic environments. Specifically, existing SLAM technologies typically assume that the environment is static, but in the real world, there are many dynamic objects (such as pedestrians, fast-moving vehicles, etc.) that adversely affect pose estimation. Therefore, the paper proposes a real-time dynamic SLAM algorithm based on the fusion of visual, inertial, and semantic information (VIS-SLAM) to improve localization accuracy and real-time performance in dynamic environments. ### Main Issues 1. **Limitations of Existing SLAM Technologies**: - **Static Environment Assumption**: Most existing SLAM technologies assume that the environment is static, which is unrealistic in practical applications. - **Impact of Dynamic Objects**: Dynamic objects (such as pedestrians, vehicles, etc.) severely affect the accuracy of pose estimation. - **Insufficient Real-time Performance**: Although deep learning algorithms are powerful, their real-time performance is poor, while pure visual geometric algorithms have poor robustness. 2. **Challenges of Localization in Dynamic Environments**: - **Identification and Handling of Dynamic Feature Points**: It is necessary to effectively identify and exclude dynamic feature points to avoid their interference with localization. - **Fusion of Multi-sensor Information**: Utilizing information from multiple sensors (such as visual, inertial, etc.) to improve the accuracy and robustness of localization. ### Solutions The paper proposes the following methods to address the above issues: 1. **Non-blocking Model Design**: - Designed a non-blocking model to extract semantic information from images, reducing the impact of the deep learning module on the real-time performance of the system and improving the execution speed of the algorithm. 2. **Motion Probability Grading and Propagation Model**: - Proposed a motion probability grading model that assigns different prior motion probabilities to objects based on different semantic types. - Designed a motion probability propagation model to estimate the prior motion probability of feature points for image frames without extracted semantic information through the feature matching relationship and semantic information of adjacent frames. 3. **Visual-Inertial Tight Coupling Feature Optimization Algorithm**: - Utilizing the tight coupling of visual and inertial measurement units (IMU), combined with prior semantic information, designed an adaptive threshold motion consistency detection algorithm to further optimize the motion probability estimation of feature points and improve localization accuracy. ### Experimental Results Experimental results show that the proposed algorithm achieves good localization accuracy in highly dynamic environments while maintaining real-time performance. ### Conclusion By fusing visual, inertial, and semantic information, this paper proposes a real-time dynamic SLAM algorithm that effectively solves the problem of autonomous localization of mobile robots in dynamic environments, improving localization accuracy and real-time performance.

A Real-Time Dynamic SLAM Algorithm Based on the Fusion of Visual, Inertial, and Semantic Information

Real-Time Dynamic SLAM Algorithm Based on Deep Learning

Dynamic Visual SLAM Based on Semantic Information and Multi-View Geometry.

A real-time semantic visual SLAM for dynamic environment based on deep learning and dynamic probabilistic propagation

Real-Time Visual-Inertial Localization Using Semantic Segmentation Towards Dynamic Environments

DRV-SLAM: An Adaptive Real-Time Semantic Visual SLAM Based on Instance Segmentation Toward Dynamic Environments

Semantic visual simultaneous localization and mapping (SLAM) using deep learning for dynamic scenes

Visual Inertial SLAM Based on Spatiotemporal Consistency Optimization in Diverse Environments

Dynamic-SLAM: Semantic monocular visual localization and mapping based on deep learning in dynamic environment

Dynamic SLAM: A Visual SLAM in Outdoor Dynamic Scenes

Visual SLAM in dynamic environments based on object detection

An adaptive visual Dynamic-SLAM method based on fusing the semantic information

Semantic information based solution for visual SLAM in dynamic environment

A semantic visual SLAM based on improved mask R-CNN in dynamic environment

A real-time, robust and versatile visual-SLAM framework based on deep learning networks

Design of visual inertial state estimator for autonomous systems via multi-sensor fusion approach

LRD-SLAM: A Lightweight Robust Dynamic SLAM Method by Semantic Segmentation Network

ADM-SLAM: Accurate and Fast Dynamic Visual SLAM with Adaptive Feature Point Extraction, Deeplabv3pro, and Multi-View Geometry

Improved Visual SLAM Algorithm Based on Dynamic Scenes