Abstract:Multi-camera-based simultaneous localization and mapping (SLAM) has been widely applied in various mobile robots under uncertain or unknown environments to accomplish tasks autonomously. However, the conventional purely data-driven feature extraction methods cannot utilize the rich semantic information in the environment, which leads to the performance of the SLAM system being susceptible to various interferences. In this work, we present a semantic-aware multi-level information fusion scheme for robust global orientation estimation. Specifically, a visual semantic perception system based on the synthesized surround view image is proposed for the multi-eye surround vision system widely used in mobile robots, which is used to obtain the visual semantic information required for SLAM tasks. The original multi-eye image was first transformed to the synthesized surround view image, and the passable space was extracted with the help of the semantic segmentation network model as a mask for feature extraction; moreover, the hybrid edge information was extracted to effectively eliminate the distorted edges by further using the distortion characteristics of the reverse perspective projection process. Then, the hybrid semantic information was used for robust global orientation estimation; thus, better localization performance was obtained. The experiments on an intelligent vehicle, which was used for automated valet parking both in indoor and outdoor scenes, showed that the proposed hybrid multi-level information fusion method achieved at least a 10-percent improvement in comparison with other edge segmentation methods, the average orientation estimation error being between 1 and 2 degrees, much smaller than other methods, and the trajectory drift value of the proposed method was much smaller than that of other methods.

Unmanned Visual Localization Based on Satellite and Image Fusion

A Robust Localization Approach Using Multi-Sensor Fusion

From Satellite to Ground: Satellite Assisted Visual Localization with Cross-view Semantic Matching

Multi-Sensor Fusion Tomato Picking Robot Localization and Mapping Research

Vehicle Localization on Semantic Map by Combining Visual and Distance Measurements

Multi-Sensor Fusion for Navigation and Mapping in Autonomous Vehicles: Accurate Localization in Urban Environments

Robust Lidar-Based Localization Scheme for Unmanned Ground Vehicle via Multisensor Fusion

Learning Visual Semantic Map-Matching for Loosely Multi-Sensor Fusion Localization of Autonomous Vehicles

A Method of Vision Aided GNSS Positioning Using Semantic Information in Complex Urban Environment

A Multisensor Fusion With Automatic Vision–LiDAR Calibration Based on Factor Graph Joint Optimization for SLAM

Driving Assistance System Based on Data Fusion of Multisource Sensors for Autonomous Unmanned Ground Vehicles.

Multi-sensor Information Fusion for Unmanned Cars Using Radar Map

Multi-UAV Collaboration and IMU Fusion Localization Method in Partial GNSS-Denied Scenarios

LiDAR-Based Sensor Fusion SLAM and Localization for Autonomous Driving Vehicles in Complex Scenarios

Real-time Geo-localization Using Satellite Imagery and Topography for Unmanned Aerial Vehicles

VL-MFL: UAV Visual Localization Based on Multisource Image Feature Learning

Off-road environment Localization of unmanned vehicles based on Visual-Inertial

Accurate Prior-centric Monocular Positioning with Offline LiDAR Fusion

Semantic-Structure-Aware Multi-Level Information Fusion for Robust Global Orientation Optimization of Autonomous Mobile Robots

A Practical Fusion Localization Method for Autonomous Vehicles in Poor GPS Areas

An Integrated Algorithm Fusing UWB Ranging Positioning and Visual–Inertial Information for Unmanned Vehicles