Abstract:The edge information plays a key role in the restoration of a depth map. Most conventional methods assume that the color image and depth map are consistent in edge areas. However, complex texture regions in the color image do not match exactly with edges in the depth map. In this paper, firstly, we point out that in most cases the consistency between normal map and depth map is much higher than that between RGB-D pairs. Then we propose a dual normal-depth regularization term to guide the restoration of depth map, which constrains the edge consistency between normal map and depth map back and forth. Moreover, considering the bimodal characteristic of weight distribution that exists in depth discontinuous areas, a reweighted graph Laplacian regularizer is proposed to promote this bimodal characteristic. And this regularization is incorporated into a unified optimization framework to effectively protect the piece-wise smoothness(PWS) characteristics of depth map. By treating depth image as graph signal, the weight between two nodes is adapted according to its content. The proposed method is tested for both noise-free and noisy cases, and is compared against the state-of-the-art methods on both synthesis and real captured datasets. Extensive experimental results demonstrate the superior performance of our method compared with most state-of-the-art works in terms of both objective and subjective quality evaluations. Specifically, our method is more effective on edge areas and more robust to noises.

Depth Upsampling Method Via Markov Random Fields Without Edge-Misaligned Artifacts

A Comparative Study on Guided Depth Enhancement

Depth map enhancement based on color and depth consistency

Guided depth enhancement via a fast marching method

Robust High Quality Image Guided Depth Upsampling

Depth Super-Resolution Using Joint Adaptive Weighted Least Squares and Patching Gradient.

Depth Map Inpainting Using a Fully Convolutional Network

Temporally Consistent Depth Map Estimation Based On 3d-Mrf

CRF-based depth refinement with hybrid depth information

Depth Map Super-Resolution and Enhancement for Time-Of-Flight Cameras

Depth Map Upsampling via Progressive Manner Based on Probability Maximization.

Depth Map Super-Resolution Via Local and Nonlocal Priors

Joint Depth Map Upsampling Via Prior Minimum Spanning Tree

Multi-Source Deep Residual Fusion Network for Depth Image Super-resolution

Guided Depth Map Super-Resolution Using Recumbent Y Network

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

Temporal Upsampling of Depth Maps Using a Hybrid Camera

Reconstructing Sparsely Represented High-resolution Depth Map from Corrupted Samples

Depth Map Super-resolution Based on Dual Normal-depth Regularization and Graph Laplacian Prior

Multiscale and Multidirection Depth Map Super Resolution with Semantic Inference

Depth Super-Resolution Via Joint Color-Guided Internal and External Regularizations