Abstract:Depth maps acquired by either physical sensors or learning methods are often seriously distorted due to boundary distortion problems, including missing, fake, and misaligned boundaries (compared with RGB images). An RGB-guided depth map recovery method is proposed in this paper to recover true boundaries in seriously distorted depth maps. Therefore, a unified model is first developed to observe all these kinds of distorted boundaries in depth maps. Observing distorted boundaries is equivalent to identifying erroneous regions in distorted depth maps, because depth boundaries are essentially formed by contiguous regions with different intensities. Then, erroneous regions are identified by separately extracting local structures of RGB image and depth map with Gaussian kernels and comparing their similarity on the basis of the SSIM index. A depth map recovery method is then proposed on the basis of the unified model. This method recovers true depth boundaries by iteratively identifying and correcting erroneous regions in recovered depth map based on the unified model and a weighted median filter. Because RGB image generally includes additional textural contents compared with depth maps, texture-copy artifacts problem is further addressed in the proposed method by restricting the model works around depth boundaries in each iteration. Extensive experiments are conducted on five RGB-depth datasets including depth map recovery, depth super-resolution, depth estimation enhancement, and depth completion enhancement. The results demonstrate that the proposed method considerably improves both the quantitative and visual qualities of recovered depth maps in comparison with fifteen competitive methods. Most object boundaries in recovered depth maps are corrected accurately, and kept sharply and well aligned with the ones in RGB images.

Depth Recovery Via Decomposition of Polynomial and Piece-Wise Constant Signals

A Depth-Adaptive Waveform Decomposition Method For Airborne Lidar Bathymetry

Depth Generation Network: Estimating Real World Depth From Stereo And Depth Images

Consistent depth maps recovery from a video sequence.

Recovering Consistent Video Depth Maps Via Bundle Optimization

Consistent Depth Maps Recovery from a Trinocular Video Sequence.

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Reconstructing Sparsely Represented High-resolution Depth Map from Corrupted Samples

Depth Upsampling Method Via Markov Random Fields Without Edge-Misaligned Artifacts

Color-Guided Depth Image Recovery with Adaptive Data Fidelity and Transferred Graph Laplacian Regularization

Depth map enhancement based on color and depth consistency

Depth Restoration from RGB-D Data Via Joint Adaptive Regularization and Thresholding on Manifolds

Depth Super-Resolution Using Joint Adaptive Weighted Least Squares and Patching Gradient.

Depth Completion using Piecewise Planar Model

Spatio-Temporal Depth Recovery of Dynamic Scenes with Multiple Handheld Cameras

Depth Map Super-resolution Based on Dual Normal-depth Regularization and Graph Laplacian Prior

Two-Stage Depth Video Recovery with Spatiotemporal Coherence

Depth Restoration: A fast low-rank matrix completion via dual-graph regularization

Depth Super-Resolution Via Joint Color-Guided Internal and External Regularizations

Depth Map Recovery Based on a Unified Depth Boundary Distortion Model

Data Driven Robust Image Guided Depth Map Restoration