Abstract:This paper studies visual odometry (VO) from the perspective of deep learning. After tremendous efforts in the robotics and computer vision communities over the past few decades, state-of-the-art VO algorithms have demonstrated incredible performance. However, since the VO problem is typically formulated as a pure geometric problem, one of the key features still missing from current VO systems is the capability to automatically gain knowledge and improve performance through learning. In this paper, we investigate whether deep neural networks can be effective and beneficial to the VO problem. An end-to-end, sequence-to-sequence probabilistic visual odometry (ESP-VO) framework is proposed for the monocular VO based on deep recurrent convolutional neural networks. It is trained and deployed in an end-to-end manner, that is, directly inferring poses and uncertainties from a sequence of raw images (video) without adopting any modules from the conventional VO pipeline. It can not only automatically learn effective feature representation encapsulating geometric information through convolutional neural networks, but also implicitly model sequential dynamics and relation for VO using deep recurrent neural networks. Uncertainty is also derived along with the VO estimation without introducing much extra computation. Extensive experiments on several datasets representing driving, flying and walking scenarios show competitive performance of the proposed ESP-VO to the state-of-the-art methods, demonstrating a promising potential of the deep learning technique for VO and verifying that it can be a viable complement to current VO systems.

Learning‐based monocular visual‐inertial odometry with SE2(3) ‐EKF

Self-supervised Visual-LiDAR Odometry with Flip Consistency

CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth

Learning Visual-Inertial Odometry With Robocentric Iterated Extended Kalman Filter

Unsupervised Monocular Visual-Inertial Odometry Network

DS-VIO: Robust and Efficient Stereo Visual Inertial Odometry based on Dual Stage EKF

Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning

High-precision, consistent EKF-based visual-inertial odometry

RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes

A self-supervised monocular odometry with visual-inertial and depth representations

Learning Interpretable BEV Based VIO without Deep Neural Networks

Optimization-Based Online Initialization and Calibration of Monocular Visual-Inertial Odometry Considering Spatial-Temporal Constraints

Beyond Learning: Back to Geometric Essence of Visual Odometry via Fusion-Based Paradigm

Pose Refinement: Bridging the Gap Between Unsupervised Learning and Geometric Methods for Visual Odometry.

Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection

End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks

Online Initialization and Extrinsic Spatial-Temporal Calibration for Monocular Visual-Inertial Odometry

Variable Observability Constrained Visual-Inertial-GNSS EKF-Based Navigation.

Learning Generalized Visual Odometry Using Position-Aware Optical Flow and Geometric Bundle Adjustment

A novel online time calibration framework via double-stage EKF for visual-inertial odometry

Robust Neural Visual Inertial Odometry with Deep Velocity Constraint