Abstract:Scene depth super-resolution (DSR) poses an inherently ill-posed problem due to the extremely large space of one-to-many mapping functions from a given low-resolution (LR) depth map, which possesses limited depth information, to multiple plausible high-resolution (HR) depth maps. This characteristic renders the task highly challenging, as identifying an optimal solution becomes significantly intricate amidst this multitude of potential mappings. While simplistic constraints have been proposed to address the DSR task, the relationship between LR and HR depth maps and the color image has not been thoroughly investigated. In this paper, we introduce a novel mapping constraint network (MCNet) that incorporates additional constraints derived from both LR depth maps and color images. This integration aims to optimize the space of mapping functions and enhance the performance of DSR. Specifically, alongside the primary DSR network (DSRNet) dedicated to learning LR-to-HR mapping, we have developed an auxiliary degradation network (ADNet) that operates in reverse, generating the LR depth map from the reconstructed HR depth map to obtain depth features in LR space. To enhance the learning process of DSRNet in LR-to-HR mapping, we introduce two mapping constraints in LR space: 1) the cycle-consistent constraint, which offers additional supervision by establishing a closed loop between LR-to-HR and HR-to-LR mappings, and 2) the region-level contrastive constraint, aimed at reinforcing region-specific HR representations by explicitly modeling the consistency between LR and HR spaces. To leverage the color image effectively, we introduce a feature screening module (FSM) to adaptively fuse color features at different layers, which can simultaneously maintain strong structural context and suppress texture distraction through subspace generation and image projection. Comprehensive experimental results across synthetic and real-world benchmark datasets unequivocally demonstrate the superiority of our proposed method over state-of-the-art DSR methods. Our MCNet achieves an average MAD reduction of 3.7% and 7.5% over state-of-the-art DSR method for \(\times\) 8 and \(\times\) 16 cases on Milddleburry dataset, respectively, without incurring additional costs during inference.

PMBANet: Progressive Multi-Branch Aggregation Network for Scene Depth Super-Resolution

Depth Image Super-resolution via Two-Branch Network

Guided Depth Map Super-Resolution Using Recumbent Y Network

Multi-Depth Branch Network for Efficient Image Super-Resolution

Super-Resolution for Monocular Depth Estimation with Multi-Scale Sub-Pixel Convolutions and a Smoothness Constraint.

BridgeNet: A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation

Progressive Multi-scale Reconstruction for Guided Depth Map Super-Resolution Via Deep Residual Gate Fusion Network

MAPANet: A Multi-Scale Attention-Guided Progressive Aggregation Network for Multi-Contrast MRI Super-Resolution

Progressive Depth Decoupling and Modulating for Flexible Depth Completion

Boundary-induced and scene-aggregated network for monocular depth prediction

MDANet: Multi-Modal Deep Aggregation Network for Depth Completion

Pyramid-Structured Depth MAP Super-Resolution Based on Deep Dense-Residual Network

Depth Super-Resolution Via Deep Controllable Slicing Network.

Hierarchical Features Driven Residual Learning for Depth Map Super-Resolution.

Degradation-Guided Multi-Modal Fusion Network for Depth Map Super-Resolution

Multiscale and Multidirection Depth Map Super Resolution with Semantic Inference

C2ANet: Cross-Scale and Cross-Modality Aggregation Network for Scene Depth Super-Resolution

Digging into Depth and Color Spaces: A Mapping Constraint Network for Depth Super-Resolution

Depth Map Super-Resolution Via Multilevel Recursive Guidance and Progressive Supervision

MSF-Net: Multi-Scale Feedback Reconstruction for Guided Depth Map Super-Resolution