Abstract:Recent RGB-guided depth super-resolution methods have achieved impressive performance under the assumption of fixed and known degradation (e.g., bicubic downsampling). However, in real-world scenarios, captured depth data often suffer from unconventional and unknown degradation due to sensor limitations and complex imaging environments (e.g., low reflective surfaces, varying illumination). Consequently, the performance of these methods significantly declines when real-world degradation deviate from their assumptions. In this paper, we propose the Degradation Oriented and Regularized Network (DORNet), a novel framework designed to adaptively address unknown degradation in real-world scenes through implicit degradation representations. Our approach begins with the development of a self-supervised degradation learning strategy, which models the degradation representations of low-resolution depth data using routing selection-based degradation regularization. To facilitate effective RGB-D fusion, we further introduce a degradation-oriented feature transformation module that selectively propagates RGB content into the depth data based on the learned degradation priors. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our DORNet in handling unknown degradation, outperforming existing methods. The code is available at <a class="link-external link-https" href="https://github.com/yanzq95/DORNet" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in real - world scenarios, the performance of Depth Super - Resolution (DSR) methods drops significantly when dealing with unknown degradations. Specifically: 1. **Limitations of existing methods**: Existing RGB - guided DSR methods usually assume that the degradation is known and fixed (for example, bicubic down - sampling), which performs well on synthetic data. However, in the real world, due to sensor limitations and complex imaging environments (such as low - reflectance surfaces, changing lighting conditions), the captured depth data often suffers from non - traditional, unknown degradations, leading to a significant decline in the performance of these methods. 2. **The core of the problem**: Degradations in the real world are diverse and unknown, and it is impossible to use labels to explicitly estimate the degradation relationship between LR depth and HR depth. In addition, this unknown degradation will lead to structural inconsistency between depth and RGB images, further affecting the restoration effect. To solve these problems, the author proposes a new framework named DORNet (Degradation Oriented and Regularized Network), aiming to adaptively handle unknown degradations in real - world scenarios through implicit degradation representations. ### Main contributions 1. **Introduction of the DORNet framework**: Use degradation representations to adaptively handle unknown degradations in real - world scenarios. 2. **Self - supervised degradation learning strategy**: Design a degradation regularization method based on routing selection to model the degradation representation of LR depth. 3. **Degradation - oriented fusion scheme**: Propose a degradation - oriented feature transformation module (DOFT) to selectively transfer RGB information into depth data according to the learned degradation prior. 4. **Experimental verification**: Extensive experimental results show that DORNet outperforms existing methods in handling unknown degradations. Through these designs, DORNet can more accurately restore high - resolution depth maps, especially when facing complex degradation situations in the real world.

Degradation Oriented and Regularized Network for Blind Depth Super-Resolution

Extreme-Quality Computational Imaging via Degradation Framework

Modular Degradation Simulation and Restoration for Under-Display Camera

Degradation-Guided Multi-Modal Fusion Network for Depth Map Super-Resolution

From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution

Unsupervised Degradation Representation Learning for Blind Super-Resolution

Difficulty-Guided Variant Degradation Learning for Blind Image Super-Resolution

Dynamic learnable degradation for blind super-resolution

Preserving Full Degradation Details for Blind Image Super-Resolution

Degradation-Guided Meta-Restoration Network for Blind Super-Resolution

DeeDSR: Towards Real-World Image Super-Resolution via Degradation-Aware Stable Diffusion

DDNSR: a dual-input degradation network for real-world super-resolution

Self-Supervised Learning for RGB-Guided Depth Enhancement by Exploiting the Dependency Between RGB and Depth

Generation diffusion degradation: Simple and efficient design for blind super-resolution

Adaptive Blind Super-Resolution Network for Spatial-Specific and Spatial-Agnostic Degradations

RestoreDet: Degradation Equivariant Representation for Object Detection in Low Resolution Images

Meta-Learning based Degradation Representation for Blind Super-Resolution

Knowledge Distillation based Degradation Estimation for Blind Super-Resolution

Efficient blind super-resolution imaging via adaptive degradation-aware estimation

DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration

Meta-learning based blind image super-resolution approach to different degradations