Abstract:Estimating scene depth from a single image can be widely applied to understand 3D environments due to the easy access of the images captured by consumer-level cameras. Previous works exploit conditional random fields (CRFs) to estimate image depth, where neighboring pixels (superpixels) with similar appearances are constrained to share the same depth. However, the depth may vary significantly in the slanted surface, thus leading to severe estimation errors. In order to eliminate those errors, we propose a superpixel-based normal guided scale invariant deep convolutional field by encouraging the neighboring superpixels with similar appearance to lie on the same 3D plane of the scene. In doing so, a depth-normal multitask CNN is introduced to produce the superpixel-wise depth and surface normal predictions simultaneously. To correct the errors of the roughly estimated superpiexl-wise depth, we develop a normal guided scale invariant CRF (NGSI-CRF). NGSI-CRF consists of a scale invariant unary potential that is able to measure the relative depth between superpixels as well as the absolute depth of superpixels, and a normal guided pairwise potential that constrains spatial relationships between superpixels in accordance with the 3D layout of the scene. In other words, the normal guided pairwise potential is designed to smooth the depth prediction without deteriorating the 3D structure of the depth prediction. The superpixel-wise depth maps estimated by NGSI-CRF will be fed into a pixel-wise refinement module to produce a smooth fine-grained depth prediction. Furthermore, we derive a closed-form solution for the maximum a posteriori (MAP) inference of NGSI-CRF. Thus, our proposed network can be efficiently trained in an end-to-end manner. We conduct our experiments on various datasets, such as NYU-D2, KITTI, and Make 3D. As demonstrated in the experimental results, our method achieves superior performance in both indoor and outdoor scenes.

Occlusion-aware Unsupervised Light Field Depth Estimation based on Muti-Scale GANs

A Learning-Based Method Using Epipolar Geometry for Light Field Depth Estimation

Depth Generation Network: Estimating Real World Depth From Stereo And Depth Images

Unsupervised Learning of Depth Estimation and Camera Pose With Multi-Scale GANs

Monocular Depth Estimation Based on Multi-Scale Graph Convolution Networks

Depth-Conditioned GAN for Underwater Image Enhancement

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Unsupervised Depth Estimation from Light Field Using a Convolutional Neural Network

Light-field-depth-estimation Network Based on Epipolar Geometry and Image Segmentation.

Unsupervised Dense Light Field Reconstruction with Occlusion Awareness

Generative Adversarial Networks for Unsupervised Monocular Depth Prediction

Depth Estimation from Monocular Image and Coarse Depth Points Based on Conditional GAN

Conditional Generative Adversarial Network for Monocular Image Depth Map Prediction

Self-supervised monocular depth estimation on construction sites in low-light conditions and dynamic scenes

Unsupervised Monocular Depth Estimation Based on Hierarchical Feature-Guided Diffusion

DepthGAN: GAN-based depth generation from semantic layouts

OccCasNet: Occlusion-aware Cascade Cost Volume for Light Field Depth Estimation

Occlusion-Model Guided Anti-Occlusion Depth Estimation in Light Field

Unsupervised Learning of Depth and Depth-of-Field Effect from Natural Images with Aperture Rendering Generative Adversarial Networks

Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields

Occlusion-Aware Depth Estimation with Adaptive Normal Constraints