Abstract:This paper studies visual odometry (VO) from the perspective of deep learning. After tremendous efforts in the robotics and computer vision communities over the past few decades, state-of-the-art VO algorithms have demonstrated incredible performance. However, since the VO problem is typically formulated as a pure geometric problem, one of the key features still missing from current VO systems is the capability to automatically gain knowledge and improve performance through learning. In this paper, we investigate whether deep neural networks can be effective and beneficial to the VO problem. An end-to-end, sequence-to-sequence probabilistic visual odometry (ESP-VO) framework is proposed for the monocular VO based on deep recurrent convolutional neural networks. It is trained and deployed in an end-to-end manner, that is, directly inferring poses and uncertainties from a sequence of raw images (video) without adopting any modules from the conventional VO pipeline. It can not only automatically learn effective feature representation encapsulating geometric information through convolutional neural networks, but also implicitly model sequential dynamics and relation for VO using deep recurrent neural networks. Uncertainty is also derived along with the VO estimation without introducing much extra computation. Extensive experiments on several datasets representing driving, flying and walking scenarios show competitive performance of the proposed ESP-VO to the state-of-the-art methods, demonstrating a promising potential of the deep learning technique for VO and verifying that it can be a viable complement to current VO systems.

Towards explainable artificial intelligence in deep vision-based odometry

Self-supervised Visual-LiDAR Odometry with Flip Consistency

Approaches, Challenges, and Applications for Deep Visual Odometry: Toward Complicated and Emerging Areas

Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas

Leveraging Deep Learning for Visual Odometry Using Optical Flow

DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometry

Salient Sparse Visual Odometry With Pose-Only Supervision

End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks

A Practical Map Needs Direct Visual Odometry

Online Indoor Visual Odometry with Semantic Assistance under Implicit Epipolar Constraints

Learning Interpretable BEV Based VIO without Deep Neural Networks

DeepVO: A Deep Learning approach for Monocular Visual Odometry

Brain-Inspired Visual Odometry: Balancing Speed and Interpretability through a System of Systems Approach

PVO: Panoptic Visual Odometry.

Learning by Inertia: Self-supervised Monocular Visual Odometry for Road Vehicles

DF-VO: What Should Be Learnt for Visual Odometry?

Reinforcement Learning Meets Visual Odometry

Visual Odometry Algorithm Based on Deep Learning

XVO: Generalized Visual Odometry via Cross-Modal Self-Training

From local understanding to global regression in monocular visual odometry

Optical Flow Assisted Monocular Visual Odometry