Abstract:Estimating scene depth from a single image can be widely applied to understand 3D environments due to the easy access of the images captured by consumer-level cameras. Previous works exploit conditional random fields (CRFs) to estimate image depth, where neighboring pixels (superpixels) with similar appearances are constrained to share the same depth. However, the depth may vary significantly in the slanted surface, thus leading to severe estimation errors. In order to eliminate those errors, we propose a superpixel-based normal guided scale invariant deep convolutional field by encouraging the neighboring superpixels with similar appearance to lie on the same 3D plane of the scene. In doing so, a depth-normal multitask CNN is introduced to produce the superpixel-wise depth and surface normal predictions simultaneously. To correct the errors of the roughly estimated superpiexl-wise depth, we develop a normal guided scale invariant CRF (NGSI-CRF). NGSI-CRF consists of a scale invariant unary potential that is able to measure the relative depth between superpixels as well as the absolute depth of superpixels, and a normal guided pairwise potential that constrains spatial relationships between superpixels in accordance with the 3D layout of the scene. In other words, the normal guided pairwise potential is designed to smooth the depth prediction without deteriorating the 3D structure of the depth prediction. The superpixel-wise depth maps estimated by NGSI-CRF will be fed into a pixel-wise refinement module to produce a smooth fine-grained depth prediction. Furthermore, we derive a closed-form solution for the maximum a posteriori (MAP) inference of NGSI-CRF. Thus, our proposed network can be efficiently trained in an end-to-end manner. We conduct our experiments on various datasets, such as NYU-D2, KITTI, and Make 3D. As demonstrated in the experimental results, our method achieves superior performance in both indoor and outdoor scenes.

Depthwise Separable Convolutions Allow for Fast and Memory-Efficient Spectral Normalization

Depth Map Inpainting Using a Fully Convolutional Network

Network Decoupling: From Regular to Depthwise Separable Convolutions

Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks

3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks

Accelerating Depthwise Separable Convolutions on Ultra-Low-Power Devices

Depthwise Multiception Convolution for Reducing Network Parameters without Sacrificing Accuracy

XSepConv: Extremely Separated Convolution

Depthwise Separable Convolutions for Neural Machine Translation

Optimizing Depthwise Separable Convolution Operations on GPUs

Depthwise-STFT based separable Convolutional Neural Networks

XSepConv: Extremely Separated Convolution for Efficient Deep Networks with Large Kernels

Factorized convolution with spectral normalization for fundus screening

Spectral Representations for Convolutional Neural Networks

Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers

Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction.

Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture

IterNorm: Fast Iterative Normalization

A Normalized Convolutional Neural Network for Guided Sparse Depth Upsampling

NSVDNet: Normalized Spatial-Variant Diffusion Network for Robust Image-Guided Depth Completion

Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields