Abstract:Despite the remarkable progresses made in deep-learning based depth map super-resolution (DSR), how to tackle real-world degradation in low-resolution (LR) depth maps remains a major challenge. Existing DSR model is generally trained and tested on synthetic dataset, which is very different from what would get from a real depth sensor. In this paper, we argue that DSR models trained under this setting are restrictive and not effective in dealing with real-world DSR tasks. We make two contributions in tackling real-world degradation of different depth sensors. First, we propose to classify the generation of LR depth maps into two types: non-linear downsampling with noise and interval downsampling, for which DSR models are learned correspondingly. Second, we propose a new framework for real-world DSR, which consists of four modules : 1) An iterative residual learning module with deep supervision to learn effective high-frequency components of depth maps in a coarse-to-fine manner; 2) A channel attention strategy to enhance channels with abundant high-frequency components; 3) A multi-stage fusion module to effectively re-exploit the results in the coarse-to-fine process; and 4) A depth refinement module to improve the depth map by TGV regularization and input loss. Extensive experiments on benchmarking datasets demonstrate the superiority of our method over current state-of-the-art DSR methods.

Multi-Direction Dictionary Learning Based Depth Map Super-Resolution with Autoregressive Modeling

Depth Map Super-Resolution Based on Joint Dictionary Learning

Depth Image Super-resolution via Two-Branch Network

Super resolution of single depth image based on multi-dictionary learning with edge feature regularization

Color Guided Depth Map Super-Resolution with Nonlocla Autoregres-Sive Modeling

Single Depth Image Super-Resolution with Multiple Residual Dictionary Learning and Refinement.

Guided Depth Map Super-Resolution Using Recumbent Y Network

Depth Map Super-Resolution Via Joint Local Gradient and Nonlocal Structural Regularizations

A Joint Learning-Based Method for Multi-view Depth Map Super Resolution

Single Image Super-Resolution by Multi-Dictionary Learning

Single Depth Map Super-Resolution Via Joint Non-Local Self-Similarity Modeling and Local Multi-Directional Gradient-Guided Regularization

A Sparse Representation Based Super-Resolution Image Reconstruction Scheme Utilizing Dual Dictionaries

Depth Map Super-Resolution Via Learned Nonlocal Model and Enhanced Local Regularization

Image Super-Resolution Based on Sparse Representation via Direction and Edge Dictionaries

Super-Resolution Reconstruction Via Multiple Sparse Dictionary Combined With Sparse Coding

Joint Nonlocal Sparse Representation for Depth Map Super-Resolution

Single Depth Map Super-Resolution Via Joint Non-Local and Local Modeling

Channel Attention based Iterative Residual Learning for Depth Map Super-Resolution

Deep Depth Super-Resolution : Learning Depth Super-Resolution using Deep Convolutional Neural Network

Superresolution Mapping Using Multiple Dictionaries by Sparse Representation

Super-resolution Reconstruction Based on Geometric Dictionary Learning and Coupled Regularization