Improving autonomous detection in dynamic environments with robust monocular thermal SLAM system
Yuzhen Wu,Lingxue Wang,Lian Zhang,Yu Bai,Yi Cai,Shuigen Wang,Yanqiu Li
DOI: https://doi.org/10.1016/j.isprsjprs.2023.08.002
IF: 12.7
2023-08-13
ISPRS Journal of Photogrammetry and Remote Sensing
Abstract:Thermal SLAM outperforms visual SLAM under conditions of nighttime, low visibility (e.g., fog, smoke, and dust), and direct glare. However, the unique non-uniformity correction for thermal imaging will cause data interruption, and the weak texture of the thermal image itself will lead to poor localization accuracy. To overcome these limitations, this paper presents a monocular thermal camera-based simultaneous localization and mapping (SLAM) system that can be used for high-precision and robust localization in challenging dynamic environments with visual degradation. We also conducted extensive experiments on small-scale indoor and outdoor sequences and on large-scale driving sequences, totaling over 180,000 real-world thermal images. Our results show that the developed MonoThermal-SLAM system can achieve camera localization and sparse structured map reconstruction in the face of visual degradation and moving objects in dynamic environments without the need for other sensors. Compared with state-of-the-art monocular SLAM systems, MonoThermal-SLAM was the only system that successfully tracked all sequences to offer unparalleled robustness and higher positioning accuracy. Relative position error (RPE) was less than 0.1 m of the ground-truth trajectory in both the indoor and outdoor sequences. Absolute trajectory error (ATE) improves averagely over 68.57% compared to other best-performing SLAM systems in driving sequences. The average relative error of tracking path length to the ground-truth was less than 5.19%. The superior performance of MonoThermal-SLAM is due to the introduction of several key innovations. First, the developed real-time scene-based denoising chain not only avoids thermal camera data interruptions but also significantly improves the thermal image quality and widens the boundaries of the application of visual SLAM systems for thermal images. Second, the combination of epipolar constraints and semantic segmentation reduces the interference of dynamic objects and improves the robustness of thermal SLAM in dynamic scenes. Third, the implemented SLAM system based on point and line features consists of well-designed initialization, tracking, local mapping, and loop closing to overcome the drawbacks of poor spatial texture distribution of thermal images. Therefore, the system can be used as a novel positioning solution to replace expensive commercial navigation systems, especially in challenging urban dynamic environments with changing light and low visibility in the air.
imaging science & photographic technology,remote sensing,geography, physical,geosciences, multidisciplinary