Abstract:The edge information plays a key role in the restoration of a depth map. Most conventional methods assume that the color image and depth map are consistent in edge areas. However, complex texture regions in the color image do not match exactly with edges in the depth map. In this paper, firstly, we point out that in most cases the consistency between normal map and depth map is much higher than that between RGB-D pairs. Then we propose a dual normal-depth regularization term to guide the restoration of depth map, which constrains the edge consistency between normal map and depth map back and forth. Moreover, considering the bimodal characteristic of weight distribution that exists in depth discontinuous areas, a reweighted graph Laplacian regularizer is proposed to promote this bimodal characteristic. And this regularization is incorporated into a unified optimization framework to effectively protect the piece-wise smoothness(PWS) characteristics of depth map. By treating depth image as graph signal, the weight between two nodes is adapted according to its content. The proposed method is tested for both noise-free and noisy cases, and is compared against the state-of-the-art methods on both synthesis and real captured datasets. Extensive experimental results demonstrate the superior performance of our method compared with most state-of-the-art works in terms of both objective and subjective quality evaluations. Specifically, our method is more effective on edge areas and more robust to noises.

Depth Map Super‐resolution Via Shape‐adaptive Non‐local Regression and Direction‐based Local Smoothness

Depth Map Super-Resolution Via Learned Nonlocal Model and Enhanced Local Regularization

Depth Map Super-Resolution Via Local and Nonlocal Priors

Depth Map Super-Resolution Via Joint Local Gradient and Nonlocal Structural Regularizations

Color Guided Depth Map Super-Resolution with Nonlocla Autoregres-Sive Modeling

Single Depth Map Super-Resolution Via Joint Non-Local Self-Similarity Modeling and Local Multi-Directional Gradient-Guided Regularization

Depth Image Super-resolution via Two-Branch Network

Single Depth Map Super-Resolution Via Joint Non-Local and Local Modeling

Color-guided Depth Map Super-Resolution Via Joint Graph Laplacian and Gradient Consistency Regularization

Joint Nonlocal Sparse Representation for Depth Map Super-Resolution

Guided Depth Map Super-Resolution Using Recumbent Y Network

Depth Super-Resolution Via Joint Color-Guided Internal and External Regularizations

Depth Map Super-Resolution Based on Joint Dictionary Learning

Depth Map Super-resolution Based on Dual Normal-depth Regularization and Graph Laplacian Prior

Image Guided Depth Map Superresolution Using Non-Local Total Generalized Variation

Depth Super-Resolution Using Joint Adaptive Weighted Least Squares and Patching Gradient.

Single Depth Map Super-resolution with Local Self-similarity

Depth Super-Resolution from RGB-D Pairs with Transform and Spatial Domain Regularization.

Depth Map Super-Resolution Via Low-Resolution Depth Guided Joint Trilateral Up-Sampling

Multiscale and Multidirection Depth Map Super Resolution with Semantic Inference

Deep Depth Super-Resolution : Learning Depth Super-Resolution using Deep Convolutional Neural Network