Abstract:One of the most challenging issues in color guided depth map restoration is the inconsistency between color edges in guidance color images and depth discontinuities on depth maps. This makes the restored depth map suffer from texture copy artifacts and blurring depth discontinuities. To handle this problem, most state-of-the-art methods design complex guidance weight based on guidance color images and heuristically make use of the bicubic interpolation of the input depth map. In this paper, we show that using bicubic interpolated depth map can blur depth discontinuities when the upsampling factor is large and the input depth map contains large holes and heavy noise. In contrast, we propose a robust optimization framework for color guided depth map restoration. By adopting a robust penalty function to model the smoothness term of our model, we show that the proposed method is robust against the inconsistency between color edges and depth discontinuities even when we use simple guidance weight. To the best of our knowledge, we are the first to solve this problem with a principled mathematical formulation rather than previous heuristic weighting schemes. The proposed robust method performs well in suppressing texture copy artifacts. Moreover, it can better preserve sharp depth discontinuities than previous heuristic weighting schemes. Through comprehensive experiments on both simulated data and real data, we show promising performance of the proposed method.

An improved MRF model for robust color guided depth up-sampling

A Comparative Study on Guided Depth Enhancement

An MRF-Based Depth Upsampling: Upsample the Depth Map with Its Own Property

Depth Upsampling Method Via Markov Random Fields Without Edge-Misaligned Artifacts

Guided depth enhancement via a fast marching method

High-quality Depth Up-Sampling Via a Supervised Classification Guided MRF Model.

Depth map enhancement based on color and depth consistency

A revisit to MRF-based depth map super-resolution and enhancement

Robust High Quality Image Guided Depth Upsampling

Color-guided Optimization Model with Reliable Self-Structure Priors for Depth Map Restoration

Robust Color Guided Depth Map Restoration.

Depth Super-Resolution Using Joint Adaptive Weighted Least Squares and Patching Gradient.

Depth Map Upsampling Using Segmentation and Edge Information.

RGB-Guided Depth Map Recovery by Two-Stage Coarse-to-Fine Dense CRF Models

Misaligned RGB-Depth Boundary Identification and Correction for Depth Image Recovery

Temporally Consistent Depth Map Estimation Based On 3d-Mrf

An Improved Mixture of Gaussians Model Based on Color and Depth Data

Edge-Guided Depth Map Enhancement

Color-guided Depth Map Super-Resolution Via Joint Graph Laplacian and Gradient Consistency Regularization

Color Guided Depth Map Super-Resolution with Nonlocla Autoregres-Sive Modeling

Depth Image Rectification Based on an Effective RGB–Depth Boundary Inconsistency Model