Abstract:Estimating scene depth from a single image can be widely applied to understand 3D environments due to the easy access of the images captured by consumer-level cameras. Previous works exploit conditional random fields (CRFs) to estimate image depth, where neighboring pixels (superpixels) with similar appearances are constrained to share the same depth. However, the depth may vary significantly in the slanted surface, thus leading to severe estimation errors. In order to eliminate those errors, we propose a superpixel-based normal guided scale invariant deep convolutional field by encouraging the neighboring superpixels with similar appearance to lie on the same 3D plane of the scene. In doing so, a depth-normal multitask CNN is introduced to produce the superpixel-wise depth and surface normal predictions simultaneously. To correct the errors of the roughly estimated superpiexl-wise depth, we develop a normal guided scale invariant CRF (NGSI-CRF). NGSI-CRF consists of a scale invariant unary potential that is able to measure the relative depth between superpixels as well as the absolute depth of superpixels, and a normal guided pairwise potential that constrains spatial relationships between superpixels in accordance with the 3D layout of the scene. In other words, the normal guided pairwise potential is designed to smooth the depth prediction without deteriorating the 3D structure of the depth prediction. The superpixel-wise depth maps estimated by NGSI-CRF will be fed into a pixel-wise refinement module to produce a smooth fine-grained depth prediction. Furthermore, we derive a closed-form solution for the maximum a posteriori (MAP) inference of NGSI-CRF. Thus, our proposed network can be efficiently trained in an end-to-end manner. We conduct our experiments on various datasets, such as NYU-D2, KITTI, and Make 3D. As demonstrated in the experimental results, our method achieves superior performance in both indoor and outdoor scenes.

Single image depth estimation by dilated deep residual convolutional neural network and soft-weight-sum inference

Dilated Fully Convolutional Neural Network for Depth Estimation from a Single Image

Monocular image depth estimation using dilated convolution and spatial pyramid polling structure

Binocular Depth Estimation Using Convolutional Neural Network With Siamese Branches.

Monocular Depth Estimation Based on Multi-Scale Graph Convolution Networks

Depth Map Inpainting Using a Fully Convolutional Network

Monocular depth estimation with hierarchical fusion of dilated CNNs and soft-weighted-sum inference

Monocular Depth Estimation Based on Dilated Convolutions and Feature Fusion

A Cnn-Based Depth Estimation Approach With Multi-Scale Sub-Pixel Convolutions And A Smoothness Constraint

Residual Feature Pyramid Architecture for Monocular Depth Estimation.

Depth Estimation from Monocular Images Using Dilated Convolution and Uncertainty Learning.

Depth Inference with Convolutional Neural Network.

Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields

Depth Reconstruction from Single Images Using a Convolutional Neural Network and a Condition Random Field Model.

Fully Convolutional Multi-Scale Dense Networks for Monocular Depth Estimation.

Enhanced Monocular Depth Estimation: A CNN Integrating Semantic Segmentation Embedding And Vanishing Point Detection

Deep Convolutional Neural Fields for Depth Estimation from a Single Image

Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields

Monocular Indoor Depth Estimation Method Based on Neural Networks with Constraints on Two-Dimensional Images and Three-Dimensional Geometry

Monocular Depth Estimation Based on Residual Pooling and Global-Local Feature Fusion

Single Image Super-resolution via a Lightweight Residual Convolutional Neural Network