Abstract:This paper addresses the self-supervised learning of several critical factors in Visual Simultaneous Localization and Mapping (VSLAM) in low-level vision: interest point learning, descriptor learning, ego-motion estimation, and depth estimation. The key insight we have is that appearance and geometry constraints can be used to couple these fundamental vision issues. We propose a self-supervised framework for joint training of neural networks for multiple objectives to address complicated issues, simplify systems, and provide important information for deep monocular VSLAM systems. First, we input two adjacent images into pose and depth networks to obtain their corresponding depth maps and camera poses. Then, we employ a differentiable geometry module and utilize the depth maps and camera poses to generate pseudo-input images needed for the interest point network and construct the geometry loss. Further, we input the pseudo-input image and source image into the interest point network to obtain the corresponding interest points, descriptors, and scores. Subsequently, we construct the appearance loss. Finally, we combine the geometry and appearance losses to constrain the whole network in an unsupervised manner. The novelty of this paper is that it integrates the key information necessary in monocular VSLAM into a unified framework that takes into account interest point learning, descriptor learning, ego-motion estimation, and depth estimation at the same time. Without providing any ground truth, our model can combine sub-problems for self-supervised learning and achieve state-of-the-art performance in their respective domains.

Keypoint Heatmap Guided Self-Supervised Monocular Visual Odometry

Self-supervised Visual-LiDAR Odometry with Flip Consistency

Unsupervised Monocular Visual Odometry Based on Confidence Evaluation

Self-Supervised Learning of Monocular Visual Odometry and Depth with Uncertainty-Aware Scale Consistency

Using Unsupervised Deep Learning Technique for Monocular Visual Odometry.

Self-supervised Visual Odometry Based on Geometric Consistency

Self-Supervised Deep Visual Odometry Based on Geometric Attention Model

Unsupervised Collaborative Learning of Keyframe Detection and Visual Odometry Towards Monocular Deep SLAM.

Unsupervised Deep Learning-Based RGB-D Visual Odometry

Robust self-supervised monocular visual odometry based on prediction-update pose estimation network.

Self-Supervised monocular visual odometry based on cross-correlation

Using Self-Supervised Depth and Uncertainty Predictions for Monocular Visual Odometry.

Self-Supervised Geometry-Guided Initialization for Robust Monocular Visual Odometry

Self-Improving Visual Odometry

Joint Self-Supervised Learning of Interest Point, Descriptor, Depth, and Ego-Motion from Monocular Video

Unsupervised Monocular Visual-Inertial Odometry Network

A Monocular Visual-Inertial Odometry Based on Hybrid Residuals.

Visual Odometry Algorithm Based on Deep Learning

Guided Feature Selection for Deep Visual Odometry

An Unsupervised Monocular Visual Odometry Based on Multi-Scale Modeling

Salient Sparse Visual Odometry With Pose-Only Supervision