Abstract:Monocular visual odometry (VO) has attracted extensive research attention by providing real-time vehicle motion from cost-effective camera images. However, state-of-the-art optimization-based monocular VO methods suffer from the scale inconsistency problem for long-term predictions. Deep learning has recently been introduced to address this issue by leveraging stereo sequences or ground-truth motions in the training dataset. However, it comes at an additional cost for data collection, and such training data may not be available in all datasets. In this work, we propose VRVO, a novel framework for retrieving the absolute scale from virtual data that can be easily obtained from modern simulation environments, whereas in the real domain no stereo or ground-truth data are required in either the training or inference phases. Specifically, we first train a scale-aware disparity network using both monocular real images and stereo virtual data. The virtual-to-real domain gap is bridged by using an adversarial training strategy to map images from both domains into a shared feature space. The resulting scale-consistent disparities are then integrated with a direct VO system by constructing a virtual stereo objective that ensures the scale consistency over long trajectories. Additionally, to address the suboptimality issue caused by the separate optimization backend and the learning process, we further propose a mutual reinforcement pipeline that allows bidirectional information flow between learning and optimization, which boosts the robustness and accuracy of each other. We demonstrate the effectiveness of our framework on the KITTI and vKITTI2 datasets.

Self-Supervised Learning of Monocular Visual Odometry and Depth with Uncertainty-Aware Scale Consistency

Self-supervised Visual-LiDAR Odometry with Flip Consistency

Self-supervised Monocular Depth and Visual Odometry Learning with Scale-consistent Geometric Constraints

Monocular Depth Estimation Based on Unsupervised Learning

SelfOdom: Self-supervised Egomotion and Depth Learning via Bi-directional Coarse-to-Fine Scale Recovery

Unsupervised Learning of Depth and Ego-Motion with Absolutely Global Scale Recovery from Visual and Inertial Data Sequences

Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics

An Unsupervised Monocular Visual Odometry Based on Multi-Scale Modeling

Depth Estimation from Monocular Images Using Dilated Convolution and Uncertainty Learning.

Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World

Monocular Depth Estimation Using Self-Supervised Learning with More Effective Geometric Constraints

Unsupervised Monocular Visual Odometry Based on Confidence Evaluation

Self-supervised deep monocular visual odometry and depth estimation with observation variation

Accurate and Robust Scale Recovery for Monocular Visual Odometry Based on Plane Geometry

Monocular Depth and Ego-motion Estimation with Scale Based on Superpixel and Normal Constraints

Unsupervised Scale-Consistent Depth Learning from Video

Towards Scale-Aware Self-Supervised Multi-Frame Depth Estimation with IMU Motion Dynamics.

Self-Supervised Scale Recovery for Monocular Depth and Egomotion Estimation

Self-Supervised Geometry-Guided Initialization for Robust Monocular Visual Odometry

Self-Supervised Monocular Depth Estimation With Self-Perceptual Anomaly Handling