Abstract:Nowadays color-guided Depth map Super-Resolution (DSR) methods mainly have three thorny problems: (1) joint DSR methods have serious detail and structure loss at very high sampling rate; (2) existing DSR networks have high computational complexity; (3) color-depth inconsistency makes it hard to fuse dual-modality features. To resolve these problems, we propose a joint hybrid-cross guidance filter method to progressively recover the quality of degraded Low-Resolution (LR) depth maps by exploiting color-depth consistency from multiple perspectives. Specifically, the proposed method leverages pyramid structure to extract multi-scale features from High-Resolution (HR) color image. At each scale, hybrid side window filter block is proposed to achieve high-efficiency color feature extraction after each down-sampling for HR color image. This block is also used to extract depth features from the LR depth map. Meanwhile, we propose a multi-perspective cross-guided fusion filter block to progressively fuse high-quality multi-scale structure information of color image with corresponding enhanced depth features. In this filter block, two kinds of space-aware group-compensation modules are introduced to capture various spatial features from different perspectives. Meanwhile, color-depth cross-attention module is proposed to extract color-depth consistency features for impactful boundary preservation. Comprehensively qualitative and quantitative experimental results have demonstrated that our method can achieve superior performances against a lot of state-of-the-art depth SR approaches in terms of mean absolute deviation and root mean square error on Middlebury, NYU-v2 and RGB-D-D datasets.

A Modified Joint Trilateral Filter Based Depth Map Refinement Method

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

Guided depth enhancement via a fast marching method

Hierarchical Joint Bilateral Filtering for Depth Post-Processing

Depth Map Optimization Based on Just Noticeable Depth Difference Model

Guided inpainting and filtering for Kinect depth maps

A Comparative Study on Guided Depth Enhancement

Depth map enhancement based on color and depth consistency

High Accuracy Hole Filling for Kinect Depth Maps

An Asymmetric Edge Adaptive Filter for Depth Generation and Hole Filling in 3DTV

Joint Trilateral Filtering For Depth Map Compression

Depth map super-resolution via iterative joint-trilateral-upsampling

Depth filter design by jointly utilizing spatial-temporal depth and texture information

Edge-Based Algorithm for Multi-view Depth Map Generation

Joint Depth Map Super-Resolution Method via Deep Hybrid-Cross Guidance Filter

Depth Map Super-Resolution Via Low-Resolution Depth Guided Joint Trilateral Up-Sampling

Candidate Value-Based Boundary Filtering for Compressed Depth Images

3D-Spatial-texture Bilateral Filter for Depth-Based 3D Video

Joint Depth Map Upsampling Via Prior Minimum Spanning Tree

Depth Map Super-Resolution Based on Edge-Guided Joint Trilateral Upsampling

Hole Filling Using Joint Bilateral Filtering for Moving Object Segmentation