Abstract:Self-supervised monocular depth estimation can exhibit excellent performance in static environments due to the multi-view consistency assumption during the training process. However, it is hard to maintain depth consistency in dynamic scenes when considering the occlusion problem caused by moving objects. For this reason, we propose a method of self-supervised self-distillation for monocular depth estimation (SS-MDE) in dynamic scenes, where a deep network with a multi-scale decoder and a lightweight pose network are designed to predict depth in a self-supervised manner via the disparity, motion information, and the association between two adjacent frames in the image sequence. Meanwhile, in order to improve the depth estimation accuracy of static areas, the pseudo-depth images generated by the LeReS network are used to provide the pseudo-supervision information, enhancing the effect of depth refinement in static areas. Furthermore, a forgetting factor is leveraged to alleviate the dependency on the pseudo-supervision. In addition, a teacher model is introduced to generate depth prior information, and a multi-view mask filter module is designed to implement feature extraction and noise filtering. This can enable the student model to better learn the deep structure of dynamic scenes, enhancing the generalization and robustness of the entire model in a self-distillation manner. Finally, on four public data datasets, the performance of the proposed SS-MDE method outperformed several state-of-the-art monocular depth estimation techniques, achieving an accuracy (δ1) of 89% while minimizing the error (AbsRel) by 0.102 in NYU-Depth V2 and achieving an accuracy (δ1) of 87% while minimizing the error (AbsRel) by 0.111 in KITTI.

What problem does this paper attempt to address?

The paper primarily aims to address the challenges of monocular depth estimation in dynamic scenes, particularly the difficulty in maintaining depth consistency caused by occlusions from moving objects. To solve the aforementioned issues, the authors propose a method based on self-supervised self-distillation (SS-MDE) to enhance the performance of monocular depth estimation in dynamic scenes. Specifically, this method includes the following key points: 1. **Self-Supervised Self-Distillation Framework**: This method designs a deep network that includes a multi-scale decoder and a lightweight pose network, predicting depth in a self-supervised manner through disparity, motion information, and the correlation between adjacent frames in an image sequence. Additionally, a teacher model is introduced to generate depth prior information, and a multi-view mask filtering module is designed to achieve feature extraction and noise filtering. 2. **Utilization of Pseudo Depth Images**: To improve the accuracy of depth estimation in static regions, this method uses pseudo depth images generated by the LeReS network to provide pseudo-supervision information, enhancing the refinement of depth in static regions. Meanwhile, a forgetting factor is introduced to reduce dependence on pseudo-supervision information. 3. **Multi-View Mask Filtering Module**: To improve the student model's understanding of depth structures in dynamic scenes and enhance the overall model's generalization ability and robustness, this module can generate depth and normal information of dynamic masks, thereby filtering out blurred or uncertain information produced by the teacher model. Through these technical means, the proposed SS-MDE method outperforms several advanced monocular depth estimation techniques on four public datasets, achieving 89% accuracy (δ1) on the NYU-Depth V2 dataset and minimizing the relative absolute error (AbsRel) to 0.102; achieving 87% accuracy (δ1) on the KITTI dataset and minimizing the relative absolute error (AbsRel) to 0.111. This indicates that the method can effectively improve the accuracy of monocular depth estimation in dynamic scenes.

Monocular Depth Estimation via Self-Supervised Self-Distillation

Monocular Depth Estimation Based on Unsupervised Learning

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement

MDSNet: self-supervised monocular depth estimation for video sequences using self-attention and threshold mask

Resolution-sensitive self-supervised monocular absolute depth estimation

Monocular Depth Estimation Using Self-Supervised Learning with More Effective Geometric Constraints

Self-Supervised Monocular Depth Estimation With Self-Perceptual Anomaly Handling

AggNet for Self-supervised Monocular Depth Estimation: Go an Aggressive Step Furthe.

Structure-Centric Robust Monocular Depth Estimation via Knowledge Distillation

MLDA-Net: Multi-Level Dual Attention-Based Network for Self-Supervised Monocular Depth Estimation

3D Object Aided Self-Supervised Monocular Depth Estimation

Weakly-Supervised Monocular Depth Estimationwith Resolution-Mismatched Data

Self-Supervised Monocular Depth Estimation With Multiscale Perception

Mining Supervision for Dynamic Regions in Self-Supervised Monocular Depth Estimation

SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes

Self-supervised learning monocular depth estimation from internet photos

Self-Supervised Monocular Depth Estimation Based on High-Order Spatial Interactions

A Lightweight Self-Supervised Training Framework for Monocular Depth Estimation

Self-Supervised Monocular Depth Estimation with Binary Mask and Lightweight Network

Revealing the Reciprocal Relations Between Self-Supervised Stereo and Monocular Depth Estimation