Abstract:Current, self-supervised depth estimation architectures rely on clear and sunny weather scenes to train deep neural networks. However, in many locations, this assumption is too strong. For example in the UK (2021), 149 days consisted of rain. For these architectures to be effective in real-world applications, we must create models that can generalise to all weather conditions, times of the day and image qualities. Using a combination of computer graphics and generative models, one can augment existing sunny-weather data in a variety of ways that simulate adverse weather effects. While it is tempting to use such data augmentations for self-supervised depth, in the past this was shown to degrade performance instead of improving it. In this paper, we put forward a method that uses augmentations to remedy this problem. By exploiting the correspondence between unaugmented and augmented data we introduce a pseudo-supervised loss for both depth and pose estimation. This brings back some of the benefits of supervised learning while still not requiring any labels. We also make a series of practical recommendations which collectively offer a reliable, efficient framework for weather-related augmentation of self-supervised depth from monocular video. We present extensive testing to show that our method, Robust-Depth, achieves SotA performance on the KITTI dataset while significantly surpassing SotA on challenging, adverse condition data such as DrivingStereo, Foggy CityScape and NuScenes-Night. The project website can be found here https://kieran514.github.io/Robust-Depth-Project/.

Kick Back & Relax++: Scaling Beyond Ground-Truth Depth with SlowTV & CribsTV

Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTV

Monocular Depth Estimation Based on Unsupervised Learning

Digging Into Self-Supervised Monocular Depth Estimation

Unsupervised Scale-Consistent Depth Learning from Video

SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning

Self-Supervised Learning for Monocular Depth Estimation from Aerial Imagery

Self-supervised Monocular Depth and Visual Odometry Learning with Scale-consistent Geometric Constraints

Continual Learning of Unsupervised Monocular Depth from Videos

Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics

Structure-Centric Robust Monocular Depth Estimation via Knowledge Distillation

Towards Better Data Exploitation in Self-Supervised Monocular Depth Estimation

Self-supervised learning monocular depth estimation from internet photos

Dyna-MSDepth: multi-scale self-supervised monocular depth estimation network for visual SLAM in dynamic scenes

Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera

Self-Supervised Learning based Depth Estimation from Monocular Images

EC-Depth: Exploring the consistency of self-supervised monocular depth estimation in challenging scenes

Self-supervised Monocular Depth Estimation: Let's Talk About The Weather

MonoPP: Metric-Scaled Self-Supervised Monocular Depth Estimation by Planar-Parallax Geometry in Automotive Applications

Self-Supervised Monocular Depth Estimation with Binary Mask and Lightweight Network