Abstract:Currently, simultaneous localization and mapping (SLAM) is one of the main research topics in the robotics field. Visual-inertia SLAM, which consists of a camera and an inertial measurement unit (IMU), can significantly improve robustness and enable scale weak-visibility, whereas monocular visual SLAM is scale-invisible. For ground mobile robots, the introduction of a wheel speed sensor can solve the scale weak-visibility problem and improve robustness under abnormal conditions. In this paper, a multi-sensor fusion SLAM algorithm using monocular vision, inertia, and wheel speed measurements is proposed. The sensor measurements are combined in a tightly coupled manner, and a nonlinear optimization method is used to maximize the posterior probability to solve the optimal state estimation. Loop detection and back-end optimization are added to help reduce or even eliminate the cumulative error of the estimated poses, thus ensuring global consistency of the trajectory and map. The outstanding contribution of this paper is that the wheel odometer pre-integration algorithm, which combines the chassis speed and IMU angular speed, can avoid the repeated integration caused by linearization point changes during iterative optimization; state initialization based on the wheel odometer and IMU enables a quick and reliable calculation of the initial state values required by the state estimator in both stationary and moving states. Comparative experiments were conducted in room-scale scenes, building scale scenes, and visual loss scenarios. The results showed that the proposed algorithm is highly accurate—2.2 m of cumulative error after moving 812 m (0.28%, loopback optimization disabled)—robust, and has an effective localization capability even in the event of sensor loss, including visual loss. The accuracy and robustness of the proposed method are superior to those of monocular visual inertia SLAM and traditional wheel odometers.

Multi-modal force/vision sensor fusion in 6-DOF pose tracking

MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion

VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation

Sensor-based hybrid position/force control of a robot manipulator in an uncalibrated environment

Visually Guided Model Predictive Robot Control via 6D Object Pose Localization and Tracking

Feedback Control and Sensor Fusion of Vision and Force

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

GOMSF: Graph-Optimization Based Multi-Sensor Fusion for robust UAV Pose estimation

Robust Tightly Coupled Pose Measurement Based on Multi-Sensor Fusion in Mobile Robot System

Multi-Sensor Fusion for Wheel-Inertial-Visual Systems Using a Fuzzification-Assisted Iterated Error State Kalman Filter

A novel tactile sensor with multimodal vision and tactile units for multifunctional robot interaction

Improve the Estimation of Monocular Vision 6-DOF Pose Based on the Fusion of Camera and Laser Rangefinder

A Robust 6-D Pose Tracking Approach by Fusing a Multi-Camera Tracking Device and an AHRS Module

A Robust Optical/inertial Data Fusion System for Motion Tracking of the Robot Manipulator

Proprioceptive Sensor-Based Simultaneous Multi-Contact Point Localization and Force Identification for Robotic Arms

Visual Tactile Sensor Based Force Estimation for Position-Force Teleoperation

UKF-Based Sensor Fusion for Joint-Torque Sensorless Humanoid Robots

Sensor Fusion Based Humanoid Robotic arm Control*

Tightly Coupled Algorithm of Visual Inertial Odometry and Magnetometer Fusion

MSFusion: Multilayer Sensor Fusion-Based Robust Motion Estimation

AFT-VO: Asynchronous Fusion Transformers for Multi-View Visual Odometry Estimation