Abstract:Deep convolutional neural network (DCNN) has been successfully applied to depth map super-resolution and outperforms existing methods by a wide margin. However, there still exist two major issues with these DCNN-based depth map super-resolution methods that hinder the performance: 1) the low-resolution depth maps either need to be up-sampled before feeding into the network or substantial deconvolution has to be used and 2) the supervision (high-resolution depth maps) is only applied at the end of the network, thus it is difficult to handle large up-sampling factors, such as x8 and x16. In this paper, we propose a new framework to tackle the above problems. First, we propose to represent the task of depth map super-resolution as a series of novel view synthesis sub-tasks. The novel view synthesis sub-task aims at generating (synthesizing) a depth map from a different camera pose, which could be learned in parallel. Second, to handle large up-sampling factors, we present a deeply supervised network structure to enforce strong supervision in each stage of the network. Third, a multi-scale fusion strategy is proposed to effectively exploit the feature maps at different scales and handle the blocking effect. In this way, our proposed framework could deal with challenging depth map super-resolution efficiently under large up-sampling factors (e.g., x8 and x16). Our method only uses the low-resolution depth map as input, and the support of color image is not needed, which greatly reduces the restriction of our method. Extensive experiments on various benchmarking data sets demonstrate the superiority of our method over current state-of-the-art depth map super-resolution methods.

A New Closed Loop Method of Super-Resolution for Multi-view Images.

Depth Image Super-resolution via Two-Branch Network

A Joint Learning-Based Method for Multi-view Depth Map Super Resolution

Piecewise Planar Super-Resolution for 3D Scene

Super-Resolution for Monocular Depth Estimation with Multi-Scale Sub-Pixel Convolutions and a Smoothness Constraint.

Single Depth Map Super-Resolution Via Joint Non-Local Self-Similarity Modeling and Local Multi-Directional Gradient-Guided Regularization

Single Depth Map Super-Resolution Via Joint Non-Local and Local Modeling

Depth Map Super-Resolution Using Stereo-Vision-Assisted Model

Frequency Domain Based Super-Resolution Method for Mixed-Resolution Multi-View Images

A Spatial Inter-View Auto-Regressive Super-Resolution Scheme for Multi-View Image Via Scene Matching Algorithm.

Depth Map Super-Resolution Considering View Synthesis Quality.

A Depth-Based Multi-View Super-Resolution Method Using Image Fusion and Blind Deblurring

Multi-view Stereo Combined with Space Propagation and Pixel-Level Refinement

Multiscale and Multidirection Depth Map Super Resolution with Semantic Inference

Depth Map Super-Resolution Via Joint Local Gradient and Nonlocal Structural Regularizations

Joint Nonlocal Sparse Representation for Depth Map Super-Resolution

Depth Super-resolution by Fusing Depth Imaging and Stereo Vision with Structural Determinant Information Inference

A Stereo-Vision-Assisted model for depth map super-resolution

A simultaneous method for 3D video super-resolution and high-quality depth estimation

Deeply Supervised Depth Map Super-Resolution As Novel View Synthesis.

A Multi-view Super-Resolution Method with Joint-optimization of Image Fusion and Blind Deblurring.