Abstract:Self-supervised monocular depth estimation has seen significant progress in recent years, especially in outdoor environments, i.e., autonomous driving scenes. However, depth prediction results are not satisfying in indoor scenes where most of the existing data are captured with hand-held devices. As compared to outdoor environments, estimating depth of monocular videos for indoor environments, using self-supervised methods, results in two additional challenges: (i) the depth range of indoor video sequences varies a lot across different frames, making it difficult for the depth network to induce consistent depth cues for training, whereas the maximum distance in outdoor scenes mostly stays the same as the camera usually sees the sky; (ii) the indoor sequences recorded with handheld devices often contain much more rotational motions, which cause difficulties for the pose network to predict accurate relative camera poses, while the motions of outdoor sequences are pre-dominantly translational, especially for street-scene driving datasets such as KITTI. In this work, we propose a novel framework-MonoIndoor++ by giving special considerations to those challenges and consolidating a set of good practices for improving the performance of self-supervised monocular depth estimation for indoor environments. First, a depth factorization module with transformer-based scale regression network is proposed to estimate a global depth scale factor explicitly, and the predicted scale factor can indicate the maximum depth values. Second, rather than using a single-stage pose estimation strategy as in previous methods, we propose to utilize a residual pose estimation module to estimate relative camera poses across consecutive frames iteratively. Third, to incorporate extensive coordinates guidance for our residual pose estimation module, we propose to perform coordinate convolutional encoding directly over the inputs to pose networks. The proposed method is validated on a variety of benchmark indoor datasets, i.e., EuRoC MAV, NYUv2, ScanNet and 7-Scenes, demonstrating the state-of-the-art performance. In addition, the effectiveness of each module is shown through a carefully conducted ablation study and the good generalization and universality of our trained model is also demonstrated, specifically on ScanNet and 7-Scenes datasets.

Learning Domain Invariant Features for Unsupervised Indoor Depth Estimation Adaptation

Monocular Depth Estimation Based on Unsupervised Learning

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Self-supervised Monocular Depth Estimation for All Day Images Using Domain Separation

Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation

Joint Feature-Level And Pixel-Level Domain Adaption For Object Detection In The Wild

Self-Supervised Learning of Domain Invariant Features for Depth Estimation

Learn to Adapt for Monocular Depth Estimation

Learn to Adapt for Self-Supervised Monocular Depth Estimation

Unsupervised Domain Adaptation for 3D Human Pose Estimation

Domain Adaptive Monocular Depth Estimation With Semantic Information

Unsupervised Monocular Depth Estimation in Highly Complex Environments

Deeper into Self-Supervised Monocular Indoor Depth Estimation

Monocular Depth Estimation on Adverse Weathers with Curriculum Domain Distribution Alignment

Unsupervised Domain Adaptation for 3D Keypoint Prediction from a Single Depth Scan.

Learning Depth from Monocular Videos Using Synthetic Data: A Temporally-Consistent Domain Adaptation Approach

MonoIndoor++: Towards Better Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments

Domain-invariant Representation Learning Using an Unsupervised Domain Adversarial Adaptation Deep Neural Network

Semi-Supervised Adversarial Monocular Depth Estimation

Learning Feature Decomposition for Domain Adaptive Monocular Depth Estimation

Transferring to Real-World Layouts: A Depth-aware Framework for Scene Adaptation