Abstract:Self-supervised learning of depth and ego-motion from unlabeled monocular videos has acquired promising results and drawn extensive attention. Most of the existing methods jointly train the depth and pose networks by photometric consistency of adjacent views based on the principle of structure-from-motion (SFM). However, the coupled relationship of the depth and pose networks based on the scene reprojection seriously influences the learning performance due to the scale ambiguity of image reconstruction-based geometry learning or the error accumulation between the learning-based method and multiview geometry-based method. In this article, we aim to improve the performance of depth and pose estimation without the auxiliary tasks and reduce the influence of the above problems on algorithm performance by alternatively training each task and geometric constraints from 3-D to 2-D. Distinct from jointly training the depth and pose networks, our key idea is to better utilize the mutual dependency between two tasks by alternatively training each network with respective geometric constraints while fixing the other. To make the optimization process easier, the iterative closest point (ICP)-based 3-D structural consistency-embedded epipolar geometric constraints are further introduced into depth and pose networks learning, which can take full advantage of both geometric methods. Then, a log-scale 3-D structural consistency loss is designed to put more emphasis on the smaller depth values during training. Extensive experiments on various benchmark data sets indicate the superiority of our algorithm over the state-of-the-art self-supervised methods.

SIM-Sync: From Certifiably Optimal Synchronization Over the 3D Similarity Group to Scene Reconstruction With Learned Depth

Spatio-Temporal Depth Recovery of Dynamic Scenes with Multiple Handheld Cameras

Cycle-SfM: Joint Self-Supervised Learning of Depth and Camera Motion from Monocular Image Sequences.

Deep Permutation Equivariant Structure from Motion

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

SCENES: Subpixel Correspondence Estimation With Epipolar Supervision

Towards 3D Scene Reconstruction from Locally Scale-Aligned Monocular Video Depth

Single-Frame based Deep View Synchronization for Unsynchronized Multi-Camera Surveillance

Depth-Guided Sparse Structure-from-Motion for Movies and TV Shows

3d Reconstruction Of Dynamic Scenes With Multiple Handheld Cameras

3D Reconstruction of Indoor Scenes Based on Feature and Graph Optimization

SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation

Learning to Recover 3D Scene Shape from a Single Image

A Method for 3D Reconstruction of Complex Scenes from Two Uncalibrated Views

Joint estimation of depth and motion from a monocular endoscopy image sequence using a multi-loss rebalancing network

Self-Supervised Monocular Depth Estimation With Multiscale Perception

Towards Accurate Reconstruction of 3D Scene Shape From A Single Monocular Image

An Accurate, Robust Visual Odometry and Detail-Preserving Reconstruction System

Synchronize Feature Extracting and Matching: A Single Branch Framework for 3D Object Tracking

Self-Supervised Learning of Depth and Ego-Motion from Videos by Alternative Training and Geometric Constraints from 3-D to 2-D