Abstract:Self-supervised learning methods are increasingly important for monocular depth estimation since they do not require ground-truth data during training. Although existing methods have achieved great success for better monocular depth estimation based on Convolutional Neural Networks (CNNs), the limited receptive field of CNNs usually is insufficient to effectively model the global information, e.g., relationship between foreground and background or relationship among objects, which are crucial for accurately capturing scene structure. Recently, some studies based on Transformers have attracted significant interest in computer vision. However, duo to the lack of spatial locality bias, they may fail to model the local information, e.g., fine-grained details with an image. To tackle these issues, we propose a novel self-supervised learning framework by incorporating the advantages of both the CNNs and Transformers so as to model the complete contextual information for high-quality monocular depth estimation. Specifically, the proposed method mainly includes two branches, where the Transformer branch is considered to capture the global information while the Convolution branch is exploited to preserve the local information. We also design a rectangle convolution module with pyramid structure to perceive the semi-global information, e.g. thin objects, along the horizontal and vertical directions within an image. Moreover, we propose a shape refinement module by learning the affinity matrix between pixel and its neighborhood to obtain accurate geometrical structure of scenes. Extensive experiments evaluated on KITTI, Cityscapes and Make3D dataset demonstrate that the proposed method achieves the competitive result compared with the state-of-the-art self-supervised monocular depth estimation methods and shows good cross-dataset generalization ability.

SPDET: Edge-Aware Self-Supervised Panoramic Depth Estimation Transformer With Spherical Geometry

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

PADENet: an Efficient and Robust Panoramic Monocular Depth Estimation Network for Outdoor Scenes.

GLPanoDepth: Global-to-Local Panoramic Depth Estimation

PanoFormer: Panorama Transformer for Indoor 360 Depth Estimation

DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization

E2LNet: An Efficient and Effective Lightweight Network for Panoramic Depth Estimation

Panoramic depth estimation via supervised and unsupervised learning in indoor scenes

SGFormer: Spherical Geometry Transformer for 360 Depth Estimation

Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation

PanoContext-Former: Panoramic Total Scene Understanding with a Transformer

ACDNet: Adaptively Combined Dilated Convolution for Monocular Panorama Depth Estimation

TransDSSL: Transformer Based Depth Estimation via Self-Supervised Learning

SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets

DepthFormer: Exploiting Long-range Correlation and Local Information for Accurate Monocular Depth Estimation

TAMDepth: self-supervised monocular depth estimation with transformer and adapter modulation

STS: Surround-view Temporal Stereo for Multi-view 3D Detection

Complete contextual information extraction for self-supervised monocular depth estimation

SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation

Towards Comprehensive Monocular Depth Estimation: Multiple Heads are Better Than One

Fast Monocular Depth Estimation via Side Prediction Aggregation with Continuous Spatial Refinement