Abstract:Nowadays color-guided Depth map Super-Resolution (DSR) methods mainly have three thorny problems: (1) joint DSR methods have serious detail and structure loss at very high sampling rate; (2) existing DSR networks have high computational complexity; (3) color-depth inconsistency makes it hard to fuse dual-modality features. To resolve these problems, we propose a joint hybrid-cross guidance filter method to progressively recover the quality of degraded Low-Resolution (LR) depth maps by exploiting color-depth consistency from multiple perspectives. Specifically, the proposed method leverages pyramid structure to extract multi-scale features from High-Resolution (HR) color image. At each scale, hybrid side window filter block is proposed to achieve high-efficiency color feature extraction after each down-sampling for HR color image. This block is also used to extract depth features from the LR depth map. Meanwhile, we propose a multi-perspective cross-guided fusion filter block to progressively fuse high-quality multi-scale structure information of color image with corresponding enhanced depth features. In this filter block, two kinds of space-aware group-compensation modules are introduced to capture various spatial features from different perspectives. Meanwhile, color-depth cross-attention module is proposed to extract color-depth consistency features for impactful boundary preservation. Comprehensively qualitative and quantitative experimental results have demonstrated that our method can achieve superior performances against a lot of state-of-the-art depth SR approaches in terms of mean absolute deviation and root mean square error on Middlebury, NYU-v2 and RGB-D-D datasets.

Multiscale Directional Fusion for Depth Map Super Resolution with Denoising

Multiresolution Contourlet Transform Fusion Based Depth Map Super Resolution.

Multiscale and Multidirection Depth Map Super Resolution with Semantic Inference

Super-Resolution for Monocular Depth Estimation with Multi-Scale Sub-Pixel Convolutions and a Smoothness Constraint.

Progressive Multi-scale Reconstruction for Guided Depth Map Super-Resolution Via Deep Residual Gate Fusion Network

Multi-Scale Progressive Fusion Learning for Depth Map Super-Resolution

Multi-Scale Frequency Reconstruction for Guided Depth Map Super-Resolution Via Deep Residual Network

Multi-Source Deep Residual Fusion Network for Depth Image Super-resolution

Multiscale Attention Fusion for Depth Map Super-Resolution Generative Adversarial Networks

Magnification-arbitrary Depth Super-Resolution with Multiscale Consistency Deformable Alignment

Guided Depth Map Super-Resolution Using Recumbent Y Network

Deep Color Guided Coarse-to-Fine Convolutional Network Cascade for Depth Image Super-Resolution.

Joint Depth Map Super-Resolution Method via Deep Hybrid-Cross Guidance Filter

MSF-Net: Multi-Scale Feedback Reconstruction for Guided Depth Map Super-Resolution

Lightweight Multiresolution Feature Fusion Network for Spectral Super-Resolution

Degradation-Guided Multi-Modal Fusion Network for Depth Map Super-Resolution

High-resolution Depth Maps Imaging Via Attention-based Hierarchical Multi-modal Fusion

Depth Map Continuous Super-Resolution with Local Implicit Guidance Function

Depth Map Super-Resolution Via Low-Resolution Depth Guided Joint Trilateral Up-Sampling

WAFP-Net: Weighted Attention Fusion based Progressive Residual Learning for Depth Map Super-resolution

Single Depth Map Super-Resolution Via Joint Non-Local Self-Similarity Modeling and Local Multi-Directional Gradient-Guided Regularization