Abstract:Estimating scene depth from a single image can be widely applied to understand 3D environments due to the easy access of the images captured by consumer-level cameras. Previous works exploit conditional random fields (CRFs) to estimate image depth, where neighboring pixels (superpixels) with similar appearances are constrained to share the same depth. However, the depth may vary significantly in the slanted surface, thus leading to severe estimation errors. In order to eliminate those errors, we propose a superpixel-based normal guided scale invariant deep convolutional field by encouraging the neighboring superpixels with similar appearance to lie on the same 3D plane of the scene. In doing so, a depth-normal multitask CNN is introduced to produce the superpixel-wise depth and surface normal predictions simultaneously. To correct the errors of the roughly estimated superpiexl-wise depth, we develop a normal guided scale invariant CRF (NGSI-CRF). NGSI-CRF consists of a scale invariant unary potential that is able to measure the relative depth between superpixels as well as the absolute depth of superpixels, and a normal guided pairwise potential that constrains spatial relationships between superpixels in accordance with the 3D layout of the scene. In other words, the normal guided pairwise potential is designed to smooth the depth prediction without deteriorating the 3D structure of the depth prediction. The superpixel-wise depth maps estimated by NGSI-CRF will be fed into a pixel-wise refinement module to produce a smooth fine-grained depth prediction. Furthermore, we derive a closed-form solution for the maximum a posteriori (MAP) inference of NGSI-CRF. Thus, our proposed network can be efficiently trained in an end-to-end manner. We conduct our experiments on various datasets, such as NYU-D2, KITTI, and Make 3D. As demonstrated in the experimental results, our method achieves superior performance in both indoor and outdoor scenes.

SuperDepthTransfer: Depth Extraction from Image Using Instance-Based Learning with Superpixels

Depth Estimation from Multi-Scale SLIC Superpixels Using Non-Parametric Learning

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Depth Image Super-resolution via Two-Branch Network

Depth Map Inpainting Using a Fully Convolutional Network

Guided Depth Map Super-Resolution Using Recumbent Y Network

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Depth Extraction Method Based on Multi-view Stereo Matching

Deep Depth Super-Resolution : Learning Depth Super-Resolution using Deep Convolutional Neural Network

Depth Estimation from Monocular Images Using Dilated Convolution and Uncertainty Learning.

Multiscale and Multidirection Depth Map Super Resolution with Semantic Inference

Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields

Depth Super-Resolution Using Joint Adaptive Weighted Least Squares and Patching Gradient.

Depth Map Generation for 2D-to-3d Conversion by Limited User Inputs and Depth Propagation

Depth Restoration with Normal-Guided Multiresolution Superpixel

DepthFormer: Exploiting Long-range Correlation and Local Information for Accurate Monocular Depth Estimation

Joint Depth Map Super-Resolution Method via Deep Hybrid-Cross Guidance Filter

Synthetic Depth Transfer for Monocular 3D Object Pose Estimation in the Wild.

Depth Is All You Need for Monocular 3D Detection

Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios

Depth Super-Resolution Via Joint Color-Guided Internal and External Regularizations