Abstract:Depth maps acquired by either physical sensors or learning methods are often seriously distorted due to boundary distortion problems, including missing, fake, and misaligned boundaries (compared with RGB images). An RGB-guided depth map recovery method is proposed in this paper to recover true boundaries in seriously distorted depth maps. Therefore, a unified model is first developed to observe all these kinds of distorted boundaries in depth maps. Observing distorted boundaries is equivalent to identifying erroneous regions in distorted depth maps, because depth boundaries are essentially formed by contiguous regions with different intensities. Then, erroneous regions are identified by separately extracting local structures of RGB image and depth map with Gaussian kernels and comparing their similarity on the basis of the SSIM index. A depth map recovery method is then proposed on the basis of the unified model. This method recovers true depth boundaries by iteratively identifying and correcting erroneous regions in recovered depth map based on the unified model and a weighted median filter. Because RGB image generally includes additional textural contents compared with depth maps, texture-copy artifacts problem is further addressed in the proposed method by restricting the model works around depth boundaries in each iteration. Extensive experiments are conducted on five RGB-depth datasets including depth map recovery, depth super-resolution, depth estimation enhancement, and depth completion enhancement. The results demonstrate that the proposed method considerably improves both the quantitative and visual qualities of recovered depth maps in comparison with fifteen competitive methods. Most object boundaries in recovered depth maps are corrected accurately, and kept sharply and well aligned with the ones in RGB images.

Depth Image Rectification Based on an Effective RGB–Depth Boundary Inconsistency Model

Misaligned RGB-Depth Boundary Identification and Correction for Depth Image Recovery

Depth Map Recovery Based on a Unified Depth Boundary Distortion Model

Depth map enhancement based on color and depth consistency

Guided depth enhancement via a fast marching method

Depth Error Elimination for RGB-D Cameras

Modeling Deviations of Rgb-D Cameras for Accurate Depth Map and Color Image Registration.

A non-parametric depth modification model for registration between color and depth images

Calibration of RGB-D Camera Using Depth Correction Model

Depth incorporating with color improves salient object detection

An improved MRF model for robust color guided depth up-sampling

A Unified Model For Improving Depth Accuracy In Kinect Sensor

RGB-Depth Structure Similarity for Self-supervised Monocular Depth Estimation

RGB-Guided Depth Map Recovery by Two-Stage Coarse-to-Fine Dense CRF Models

A Post-Rectification Approach of Depth Images of Kinect V2 for 3D Reconstruction of Indoor Scenes

Edge-Guided Depth Map Enhancement

RGB-D depth-map restoration using smooth depth neighborhood supports

Shape Preserving RGB-D Depth Map Restoration.

Depth-guided Saliency Detection Via Boundary Information

Single View 3D Reconstruction Based on Improved RGB-D Image

Delving into Calibrated Depth for Accurate RGB-D Salient Object Detection