Abstract:Although depth cameras acquire depth in dynamic scenes, the captured depth images are often noisy and of low resolution. Depth images have the physical nature of being represented by smooth regions and edges in between them, i.e. depth images have high edge sparsity in the gradient domain. In this paper, we propose intensity guided depth upsampling using edge sparsity and super-weighted L 0 gradient minimization. First, we get mutual structure between intensity and depth using joint mutual structure filtering. Second, we generate an initial depth image using recursive interpolation. Next, we generate weights for L 0 gradient minimization based on gradient and entropy of the intensity image, and upsample the depth image using super-weighted L 0 gradient minimization. In super-weighted L 0 gradient minimization, we combine two main terms: Hybrid data fidelity and weighted L 0 gradient regularization. The hybrid data fidelity term combines both zero-order and first-order data differences to suppress staircase artifacts, while the weighted L 0 gradient regularization term preserves depth structures and removes noise. Finally, we further refine the depth image using adaptive fast weighted median filtering. Experiments on Middlebury and realworld scene datasets verify that the proposed method produces edge preserving depth upsampling results and outperforms state-of-the-arts in terms of both visual quality and quantitative measurements.

Non-convex joint bilateral guided depth upsampling

A Comparative Study on Guided Depth Enhancement

Depth Edge Guided CNNs for Sparse Depth Upsampling

Robust High Quality Image Guided Depth Upsampling

Guided depth enhancement via a fast marching method

Depth Super-Resolution Using Joint Adaptive Weighted Least Squares and Patching Gradient.

Intensity Guided Depth Upsampling Using Edge Sparsity and Super-Weighted L Gradient Minimization.

Depth map super-resolution via iterative joint-trilateral-upsampling

Robust Color Guided Depth Map Restoration.

Boundary-Preserving Depth Upsampling Without Texture Copying Artifacts and Holes

Upsampling The Depth Map With Its Own Properties

Intensity-Guided Depth Upsampling Using Edge Sparsity and WeightedL0Gradient Minimization.

Joint Bilateral Upsampling

Intensity-guided Edge-Preserving Depth Upsampling Through Weighted L 0 Gradient Minimization

Hierarchical Joint Bilateral Filtering for Depth Post-Processing

Intensity-Guided Depth Upsampling Using Edge Sparsity and Weighted <tex>$L_{0}$</tex> Gradient Minimization

An MRF-Based Depth Upsampling: Upsample the Depth Map with Its Own Property

Fast Multi-Scale Joint Bilateral Texture Upsampling

Robust Weighted Least Squares For Guided Depth Upsampling

Guided Linear Upsampling

Color-guided Boundary-Preserving Depth Upsampling Based on L0 Gradient Minimization.