Abstract:Depth estimation from a single image is an active research topic in computer vision. The most accurate approaches are based on fully supervised learning models, which rely on a large amount of dense and high-resolution (HR) ground-truth depth maps. However, in practice, color images are usually captured with much higher resolution than depth maps, leading to the resolution-mismatched effect. In this paper, we propose a novel weakly-supervised framework to train a monocular depth estimation network to generate HR depth maps with resolution-mismatched supervision, i.e., the inputs are HR color images and the ground-truth are low-resolution (LR) depth maps. The proposed weakly supervised framework is composed of a sharing weight monocular depth estimation network and a depth reconstruction network for distillation. Specifically, for the monocular depth estimation network the input color image is first downsampled to obtain its LR version with the same resolution as the ground-truth depth. Then, both HR and LR color images are fed into the proposed monocular depth estimation network to obtain the corresponding estimated depth maps. We introduce three losses to train the network: 1) reconstruction loss between the estimated LR depth and the ground-truth LR depth; 2) reconstruction loss between the downsampled estimated HR depth and the ground-truth LR depth; 3) consistency loss between the estimated LR depth and the downsampled estimated HR depth. In addition, we design a depth reconstruction network from depth to depth. Through distillation loss, features between two networks maintain the structural consistency in affinity space, and finally improving the estimation network performance. Experimental results demonstrate that our method achieves superior performance than unsupervised and semi-supervised learning based schemes, and is competitive or even better compared to supervised ones.

Depth Discontinuity Estimation Based on Single Image

Depth map enhancement based on color and depth consistency

Depth Estimation from Monocular Images Using Dilated Convolution and Uncertainty Learning.

Reconstructing Sparsely Represented High-resolution Depth Map from Corrupted Samples

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Depth recovery from a single defocused image based on depth locally consistency

Depth Extraction Method Based on Multi-view Stereo Matching

Single-Image Depth Perception in the Wild

Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps

DDL-MVS: Depth Discontinuity Learning for MVS Networks

SuperDepthTransfer: Depth Extraction from Image Using Instance-Based Learning with Superpixels

Single Image Depth Estimation Trained via Depth from Defocus Cues

Depth Completion using Piecewise Planar Model

Depth from Defocus Via Discriminative Metric Learning

Size-to-depth: A New Perspective for Single Image Depth Estimation

Weakly-Supervised Monocular Depth Estimationwith Resolution-Mismatched Data

Depth Estimation from Multi-Scale SLIC Superpixels Using Non-Parametric Learning

Deep Monocular Depth Estimation via Integration of Global and Local Predictions

Enhanced Monocular Depth Estimation: A CNN Integrating Semantic Segmentation Embedding And Vanishing Point Detection