Abstract:The Quick-view (QV) technique serves as a primary method for detecting defects within sewerage systems. However, the effectiveness of QV is impeded by the limited visual range of its hardware, resulting in suboptimal image quality for distant portions of the sewer network. Image super-resolution is an effective way to improve image quality and has been applied in a variety of scenes. However, research on super-resolution for sewer images remains considerably unexplored. In response, this study leverages the inherent depth relationships present within QV images and introduces a novel Depth-guided, Reference-based Super-Resolution framework denoted as DSRNet. It comprises two core components: a depth extraction module and a depth information matching module (DMM). DSRNet utilizes the adjacent frames of the low-resolution image as reference images and helps them recover texture information based on the correlation. By combining these modules, the integration of depth priors significantly enhances both visual quality and performance benchmarks. Besides, in pursuit of computational efficiency and compactness, a super-resolution knowledge distillation model based on an attention mechanism is introduced. This mechanism facilitates the acquisition of feature similarity between a more complex teacher model and a streamlined student model, with the latter being a lightweight version of DSRNet. Experimental results demonstrate that DSRNet significantly improves PSNR and SSIM compared with other methods. This study also conducts experiments on sewer defect semantic segmentation, object detection, and classification on the Pipe dataset and Sewer-ML dataset. Experiments show that the method can improve the performance of low-resolution sewer images in these tasks.

What problem does this paper attempt to address?

### What problems does the paper attempt to solve? The paper aims to solve the problem of low - resolution sewer images from a long - distance perspective. Specifically: 1. **Limitations of QV devices**: Quick - view (QV) technology is the main method for detecting sewer defects. However, due to the viewing distance limitations of its hardware, the image quality of the far - distance part is poor. Such low - resolution images will hinder the effective execution of subsequent computer vision tasks (such as defect classification, object detection, and semantic segmentation). 2. **The need for super - resolution**: In order to improve the quality of low - resolution images, super - resolution (SR) technology is introduced. However, there are relatively few super - resolution studies on sewer images, and the existing general - purpose super - resolution algorithms fail to fully utilize depth information to improve the reconstruction effect. 3. **Limitations of computing resources**: QV devices have limited computing power, so it is necessary to develop lightweight models to ensure efficient deployment on these devices. ### Solutions To solve the above problems, the authors propose a sewer - image super - resolution framework based on depth priors and reference images, called DSRNet (Depth - guided, Reference - based Super - Resolution Network). This framework includes the following core components: 1. **Depth extraction module**: Generate depth maps from low - resolution images and reference images, and use depth information to guide super - resolution reconstruction. 2. **Depth information matching module (DMM)**: By combining depth - map features with features extracted by the encoder, perform feature matching to further improve the accuracy of texture restoration. 3. **Lightweight network design**: Introduce an attention - mechanism - based knowledge distillation model to transfer the knowledge of a complex teacher model to a lightweight student model, thereby reducing the model size and computing cost while maintaining performance. Through these innovations, DSRNet not only significantly improves image quality indicators such as PSNR and SSIM, but also shows better performance in tasks such as sewer defect detection. ### Formula summary - **Depth reconstruction loss**: \[ L_{dep}=\|\tilde{D}_{Ref\downarrow}-D_{Ref\downarrow}\|_1+\|\tilde{D}_{LR}-D_{LR}\|_1 \] - **Reconstruction loss**: \[ L_{rec}=\|I_{HR}-I_{SR}\|_1 \] - **Perceptual loss**: \[ L_{per}=\|\phi_i(I_{HR})-\phi_i(I_{SR})\|_2 \] - **Adversarial loss**: \[ L_G =-\mathbb{E}_{I_{HR}}[\log(1 - D(I_{HR},I_{SR}))]-\mathbb{E}_{I_{SR}}[\log(D(I_{SR},I_{HR}))] \] \[ L_D =-\mathbb{E}_{I_{HR}}[\log(D(I_{HR},I_{SR}))]-\mathbb{E}_{I_{SR}}[\log(1 - D(I_{SR},I_{HR}))] \] \[ L_{adv}=\lambda_G L_G+\lambda_D L_D \] - **Total loss function**: \[ L=\lambda_{dep}L_{dep}+\lambda_{rec}L_{rec}+\lambda_{per}L_{per}+\lambda_{adv}L_{adv} \] Through these formulas, DSRNet can effectively combine depth information and reference images to achieve high - quality super - resolution reconstruction of sewer images.

Sewer Image Super-Resolution with Depth Priors and Its Lightweight Network

Sewer defect instance segmentation, localization, and 3D reconstruction for sewer floating capsule robots

Depth Image Super-resolution via Two-Branch Network

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

Automatic sewer defect detection and severity quantification based on pixel-level semantic segmentation

Attention‐optimized 3D segmentation and reconstruction system for sewer pipelines employing multi‐view images

Guided Depth Map Super-Resolution Using Recumbent Y Network

Sewer pipe defect detection via deep learning with local and global feature fusion

Real-time sewer defect detection based on YOLO network, transfer learning, and channel pruning algorithm

SCSNet: an Efficient Paradigm for Learning Simultaneously Image Colorization and Super-resolution

Automatic Detection Method of Sewer Pipe Defects Using Deep Learning Techniques

Image Segmentation of a Sewer Based on Deep Learning

Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots

A robust instance segmentation framework for underground sewer defect detection

Remote Sensing Single-Image Resolution Improvement Using A Deep Gradient-Aware Network with Image-Specific Enhancement

Deep Learning-Based Automatic Defect Detection Method for Sewer Pipelines

Deep Learning for Subsurface Penetrating Super-Resolution Imaging

Multi-scale dense spatially-adaptive residual distillation network for lightweight underwater image super-resolution

Deep learning, geometric characterization and hydrodynamic modeling for assessing sewer defect impacts on urban flooding: A case study in Guangzhou, China

Automatic Defect Detection in Sewer Network Using Deep Learning Based Object Detector