Abstract:Nowadays color-guided Depth map Super-Resolution (DSR) methods mainly have three thorny problems: (1) joint DSR methods have serious detail and structure loss at very high sampling rate; (2) existing DSR networks have high computational complexity; (3) color-depth inconsistency makes it hard to fuse dual-modality features. To resolve these problems, we propose a joint hybrid-cross guidance filter method to progressively recover the quality of degraded Low-Resolution (LR) depth maps by exploiting color-depth consistency from multiple perspectives. Specifically, the proposed method leverages pyramid structure to extract multi-scale features from High-Resolution (HR) color image. At each scale, hybrid side window filter block is proposed to achieve high-efficiency color feature extraction after each down-sampling for HR color image. This block is also used to extract depth features from the LR depth map. Meanwhile, we propose a multi-perspective cross-guided fusion filter block to progressively fuse high-quality multi-scale structure information of color image with corresponding enhanced depth features. In this filter block, two kinds of space-aware group-compensation modules are introduced to capture various spatial features from different perspectives. Meanwhile, color-depth cross-attention module is proposed to extract color-depth consistency features for impactful boundary preservation. Comprehensively qualitative and quantitative experimental results have demonstrated that our method can achieve superior performances against a lot of state-of-the-art depth SR approaches in terms of mean absolute deviation and root mean square error on Middlebury, NYU-v2 and RGB-D-D datasets.

Hierarchical Joint Bilateral Filtering for Depth Post-Processing

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

A Comparative Study on Guided Depth Enhancement

Guided depth enhancement via a fast marching method

Guided inpainting and filtering for Kinect depth maps

Depth map enhancement based on color and depth consistency

Depth Map Optimization Based on Just Noticeable Depth Difference Model

Joint Depth Map Upsampling Via Prior Minimum Spanning Tree

Joint Depth Map Super-Resolution Method via Deep Hybrid-Cross Guidance Filter

Depth map super-resolution via iterative joint-trilateral-upsampling

A Modified Joint Trilateral Filter Based Depth Map Refinement Method

An Asymmetric Edge Adaptive Filter for Depth Generation and Hole Filling in 3DTV

Depth super resolution using bilateral filter

Candidate Value-Based Boundary Filtering for Compressed Depth Images

Depth Upsampling Method Via Markov Random Fields Without Edge-Misaligned Artifacts

Depth Image Super-resolution Reconstruction Based on Filter Fusion

Spatio-Temporal Depth Recovery of Dynamic Scenes with Multiple Handheld Cameras

Recovering Consistent Video Depth Maps Via Bundle Optimization

Efficient depth coding in 3D video to minimize coding bitrate and complexity

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Robust, Efficient Depth Reconstruction with Hierarchical Confidence-Based Matching.