Abstract:Although depth cameras acquire depth in dynamic scenes, the captured depth images are often noisy and of low resolution. Depth images have the physical nature of being represented by smooth regions and edges in between them, i.e. depth images have high edge sparsity in the gradient domain. In this paper, we propose intensity guided depth upsampling using edge sparsity and super-weighted L 0 gradient minimization. First, we get mutual structure between intensity and depth using joint mutual structure filtering. Second, we generate an initial depth image using recursive interpolation. Next, we generate weights for L 0 gradient minimization based on gradient and entropy of the intensity image, and upsample the depth image using super-weighted L 0 gradient minimization. In super-weighted L 0 gradient minimization, we combine two main terms: Hybrid data fidelity and weighted L 0 gradient regularization. The hybrid data fidelity term combines both zero-order and first-order data differences to suppress staircase artifacts, while the weighted L 0 gradient regularization term preserves depth structures and removes noise. Finally, we further refine the depth image using adaptive fast weighted median filtering. Experiments on Middlebury and realworld scene datasets verify that the proposed method produces edge preserving depth upsampling results and outperforms state-of-the-arts in terms of both visual quality and quantitative measurements.

Fast Guided Global Interpolation For Depth And Motion

A Comparative Study on Guided Depth Enhancement

Guided depth enhancement via a fast marching method

A Multiscale-Contour-based Interpolation Framework for Generating a Time-Varying Quasi-Dense Point Cloud Sequence.

SSGP: Sparse Spatial Guided Propagation for Robust and Generic Interpolation

Fast Minimax Path-Based Joint Depth Interpolation

A Hvs-Guided Approach For Real-Time Image Interpolation

Robust High Quality Image Guided Depth Upsampling

Image Interpolation Based on Non-local Geometric Similarities and Directional Gradients

Fast Binocular Depth Inference via Bidirectional Motion Based Interpolation

Guided Linear Upsampling

Edge-preserving Single Depth Image Interpolation.

Depth Super-Resolution Using Joint Adaptive Weighted Least Squares and Patching Gradient.

Saliency Guided Fast Interpolation for Large Displacement Optical Flow

Sparse Global Matching for Video Frame Interpolation with Large Motion

An Efficient 3D Video Frame Interpolation Method Using Color-Depth-motion Information

Intensity Guided Depth Upsampling Using Edge Sparsity and Super-Weighted L Gradient Minimization.

Depth-Aware Video Frame Interpolation

Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation

Temporal Upsampling of Depth Maps Using a Hybrid Camera

Bi-Directional Pseudo-Three-Dimensional Network for Video Frame Interpolation