Abstract:Monocular visual odometry (VO) has attracted extensive research attention by providing real-time vehicle motion from cost-effective camera images. However, state-of-the-art optimization-based monocular VO methods suffer from the scale inconsistency problem for long-term predictions. Deep learning has recently been introduced to address this issue by leveraging stereo sequences or ground-truth motions in the training dataset. However, it comes at an additional cost for data collection, and such training data may not be available in all datasets. In this work, we propose VRVO, a novel framework for retrieving the absolute scale from virtual data that can be easily obtained from modern simulation environments, whereas in the real domain no stereo or ground-truth data are required in either the training or inference phases. Specifically, we first train a scale-aware disparity network using both monocular real images and stereo virtual data. The virtual-to-real domain gap is bridged by using an adversarial training strategy to map images from both domains into a shared feature space. The resulting scale-consistent disparities are then integrated with a direct VO system by constructing a virtual stereo objective that ensures the scale consistency over long trajectories. Additionally, to address the suboptimality issue caused by the separate optimization backend and the learning process, we further propose a mutual reinforcement pipeline that allows bidirectional information flow between learning and optimization, which boosts the robustness and accuracy of each other. We demonstrate the effectiveness of our framework on the KITTI and vKITTI2 datasets.

Learning by Inertia: Self-supervised Monocular Visual Odometry for Road Vehicles

Self-supervised Visual-LiDAR Odometry with Flip Consistency

Attention Guided Unsupervised learning of Monocular Visual-inertial Odometry

Attention Guided Unsupervised Learning of Monocular Visualinertial Odometry

Unsupervised Monocular Visual-Inertial Odometry Network

A self-supervised monocular odometry with visual-inertial and depth representations

Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning

End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks

DeepVO: A Deep Learning approach for Monocular Visual Odometry

Beyond Learning: Back to Geometric Essence of Visual Odometry via Fusion-Based Paradigm

Learning Visual-Inertial Odometry With Robocentric Iterated Extended Kalman Filter

Self-supervised Visual Odometry Based on Geometric Consistency

Twinvo: Unsupervised Learning of Monocular Visual Odometry Using Bi-Direction Twin Network

Learning Generalized Visual Odometry Using Position-Aware Optical Flow and Geometric Bundle Adjustment

Leveraging Deep Learning for Visual Odometry Using Optical Flow

DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometry

Salient Sparse Visual Odometry With Pose-Only Supervision

Attention-based Long-term Modeling for Deep Visual Odometry

Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World

SelfVIO: Self-Supervised Deep Monocular Visual-Inertial Odometry and Depth Estimation

Learning‐based monocular visual‐inertial odometry with SE2(3) ‐EKF