Abstract:Precise, seamless, and efficient train localization as well as long‐term railway environment monitoring is the essential property towards reliability, availability, maintainability, and safety engineering for railroad systems. Simultaneous localization and mapping is right at the core of solving the two problems concurrently. To this end, we propose a high‐performance and versatile multimodal framework in this paper, targeted for the odometry and mapping task for various rail vehicles. Our system is built atop an inertial‐centric state estimator that tightly couples light detection and ranging (LiDAR), visual, optionally satellite navigation, and map‐based localization information with the convenience and extendibility of loosely coupled methods. The inertial sensors inertial measurement unit and wheel encoder are treated as the primary sensor, which achieves the observations from subsystems to constrain the accelerometer and gyroscope biases. Compared with point‐only LiDAR‐inertial methods, our approach leverages more geometry information by introducing both track plane and electric power pillars into state estimation. The visual‐inertial subsystem also utilizes the environmental structure information by employing both lines and points. Besides, the method is capable of handling sensor failures by automatic reconfiguration bypassing failure modules. Our proposed method has been extensively tested in the long‐during railway environments over 4 years, including general‐speed, high‐speed, and metro, where both passenger and freight traffic are investigated. Further, we aim to share, in an open way, the experiences, problems, and successes of our group with the robotics community so that those who work in such environments can avoid these errors. In this view, we open source some of the data sets to benefit the research community.

FRVO-Mono: Feature-Based Railway Visual Odometry With Monocular Camera

Realization of odometry system using monocular vision

Multimodal Localization: Stereo over LiDAR Map

Four years of multimodal odometry and mapping on the rail vehicles

DF-VO: What Should Be Learnt for Visual Odometry?

Experimental Comparison of Visual-Aided Odometry Methods for Rail Vehicles

Visual Localization in a Prior 3D LiDAR Map Combining Points and Lines

Rail Vehicle Localization and Mapping with LiDAR-Vision-Inertial-GNSS Fusion

Off-road environment Localization of unmanned vehicles based on Visual-Inertial

RLP‐VIO: Robust and Lightweight Plane‐based Visual‐inertial Odometry for Augmented Reality

Four years of multi-modal odometry and mapping on the rail vehicles

A self-supervised monocular odometry with visual-inertial and depth representations

Fast and Robust Monocular Visua-Inertial Odometry Using Points and Lines

ViLiVO: Virtual LiDAR-Visual Odometry for an Autonomous Vehicle with a Multi-Camera System

Improving Monocular Visual Odometry Using Learned Depth

Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World

MCVO: A Generic Visual Odometry for Arbitrarily Arranged Multi-Cameras

MS-VRO: A Multi-Stage Visual-Millimeter Wave Radar Fusion Odometry

Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences

DeepVO: A Deep Learning approach for Monocular Visual Odometry

DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometry