Abstract:This paper deals with the challenging online monocular depth adaptation task that aims to train an initial depth estimation model in a source domain and continuously adapt the model against a constantly changing target domain. Due to the high cost of real-world data collection and the camera-dependent nature of the depth estimation, previous works tend to simulate the environment using a virtual-world dataset for training (e.g. Virtual KITTI), and employ a real-world dataset for testing (e.g. KITTI, which shares the same camera settings) and are therefore vulnerable to novel domains with unknown statistics. We propose a meta-learning-based online domain adaptation framework that can leverage multi-source domain to transfer the learned knowledge from the virtual world to the real world better with the metrically accurate scale. Our learn-to-adapt algorithm mimics domain shifts during training by creating fictitious testing domains and incorporating a meta-optimization objective for optimizing the performance of the testing domains after updating the training domains in each mini-batch step. The algorithm is augmented with gradient surgery to alleviate unreliable optimization of inconsistent regions. To facilitate multi-source domain training and testing, we introduce a camera conversion technique for transforming images and depth cues from different camera settings to a unified one. During online adaptation, we also apply the exact statistics of the test-time input to the network's normalization layers to ensure a more robust adaptation. Extensive experiments demonstrate that our method can robustly adapt and outperform the virtual-to-real state-of-the-art methods on the standard KITTI Eigen benchmark of both full-length videos and isolated frames, a feat never attempted before, as well as generalizing to other real-world datasets without retraining.

Lifelong-MonoDepth: Lifelong Learning for Multi-Domain Monocular Metric Depth Estimation

Lifelong-MonoDepth: Lifelong Learning for Multidomain Monocular Metric Depth Estimation

Monocular Depth Estimation Based on Unsupervised Learning

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

A Robust Monocular Depth Estimation Framework Based on Light-Weight ERF-Pspnet for Day-Night Driving Scenes

Depth Estimation from Monocular Images Using Dilated Convolution and Uncertainty Learning.

Continual Learning of Unsupervised Monocular Depth from Videos

Learning Depth Estimation from Memory Infusing Monocular Cues: A Generalization Prediction Approach

A Lightweight Self-Supervised Training Framework for Monocular Depth Estimation

Learning Monocular Depth in Dynamic Environment via Context-aware Temporal Attention

Transfer multi-source knowledge via scale-aware online domain adaptation in depth estimation for autonomous driving

Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning

Unsupervised Monocular Depth Estimation for Monocular Visual SLAM Systems

Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics

MBUDepthNet: Real-Time Unsupervised Monocular Depth Estimation Method for Outdoor Scenes

Depth Is All You Need for Monocular 3D Detection

UniDepth: Universal Monocular Metric Depth Estimation

Learning Monocular Depth by Distilling Cross-domain Stereo Networks

G2-MonoDepth: A General Framework of Generalized Depth Inference from Monocular RGB+X Data

Structure-Centric Robust Monocular Depth Estimation via Knowledge Distillation

LAM-Depth: Laplace-Attention Module-Based Self-Supervised Monocular Depth Estimation