Abstract:Deep convolutional neural network (DCNN) has been successfully applied to depth map super-resolution and outperforms existing methods by a wide margin. However, there still exist two major issues with these DCNN-based depth map super-resolution methods that hinder the performance: 1) the low-resolution depth maps either need to be up-sampled before feeding into the network or substantial deconvolution has to be used and 2) the supervision (high-resolution depth maps) is only applied at the end of the network, thus it is difficult to handle large up-sampling factors, such as x8 and x16. In this paper, we propose a new framework to tackle the above problems. First, we propose to represent the task of depth map super-resolution as a series of novel view synthesis sub-tasks. The novel view synthesis sub-task aims at generating (synthesizing) a depth map from a different camera pose, which could be learned in parallel. Second, to handle large up-sampling factors, we present a deeply supervised network structure to enforce strong supervision in each stage of the network. Third, a multi-scale fusion strategy is proposed to effectively exploit the feature maps at different scales and handle the blocking effect. In this way, our proposed framework could deal with challenging depth map super-resolution efficiently under large up-sampling factors (e.g., x8 and x16). Our method only uses the low-resolution depth map as input, and the support of color image is not needed, which greatly reduces the restriction of our method. Extensive experiments on various benchmarking data sets demonstrate the superiority of our method over current state-of-the-art depth map super-resolution methods.

Depth Map Super-Resolution Using Synthesized View Matching for Depth-Image-Based Rendering

Depth Map Super-Resolution Considering View Synthesis Quality.

Synthesis-guided depth super resolution

Guided Depth Map Super-Resolution Using Recumbent Y Network

Depth Map Super-Resolution Based on Joint Dictionary Learning

Depth-image-based Rendering with Spatial and Temporal Texture Synthesis for 3DTV

An Efficient Depth Image-Based Rendering with Depth Reliability Maps for View Synthesis

Depth Image Super-Resolution Reconstruction Based on a Modified Joint Trilateral Filter.

Improved Multi-View Depth Estimation For View Synthesis In 3d Video Coding

Depth-image Based View Synthesis for Threedimensional Television

Multiscale and Multidirection Depth Map Super Resolution with Semantic Inference

Depth Map Compression Using Multi-Resolution Graph-Based Transform for Depth-Image-Based Rendering

Depth Map Super-Resolution Via Low-Resolution Depth Guided Joint Trilateral Up-Sampling

Depth Map Continuous Super-Resolution with Local Implicit Guidance Function

Joint Nonlocal Sparse Representation for Depth Map Super-Resolution

Channel Attention based Iterative Residual Learning for Depth Map Super-Resolution

Depth map super-resolution via iterative joint-trilateral-upsampling

Depth Map Super-Resolution Using Stereo-Vision-Assisted Model

Depth Image Super Resolution Based on Edge-Guided Method

Deeply Supervised Depth Map Super-Resolution As Novel View Synthesis.

WLDISR: Weighted Local Sparse Representation-Based Depth Image Super-Resolution for 3D Video System