Target-oriented Domain Adaptation for Infrared Image Super-Resolution

Yongsong Huang,Tomo Miyazaki,Xiaofeng Liu,Yafei Dong,Shinichiro Omachi

2023-11-15

Abstract:Recent efforts have explored leveraging visible light images to enrich texture details in infrared (IR) super-resolution. However, this direct adaptation approach often becomes a double-edged sword, as it improves texture at the cost of introducing noise and blurring artifacts. To address these challenges, we propose the Target-oriented Domain Adaptation SRGAN (DASRGAN), an innovative framework specifically engineered for robust IR super-resolution model adaptation. DASRGAN operates on the synergy of two key components: 1) Texture-Oriented Adaptation (TOA) to refine texture details meticulously, and 2) Noise-Oriented Adaptation (NOA), dedicated to minimizing noise transfer. Specifically, TOA uniquely integrates a specialized discriminator, incorporating a prior extraction branch, and employs a Sobel-guided adversarial loss to align texture distributions effectively. Concurrently, NOA utilizes a noise adversarial loss to distinctly separate the generative and Gaussian noise pattern distributions during adversarial training. Our extensive experiments confirm DASRGAN's superiority. Comparative analyses against leading methods across multiple benchmarks and upsampling factors reveal that DASRGAN sets new state-of-the-art performance standards. Code are available at \url{<a class="link-external link-https" href="https://github.com/yongsongH/DASRGAN" rel="external noopener nofollow">this https URL</a>}.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the paper attempts to solve

The paper "Target - oriented Domain Adaptation for Infrared Image Super - Resolution" aims to solve several key problems in infrared image super - resolution (IR - SR):

1. **Texture detail enhancement**:
- Existing methods often introduce noise and blurring artifacts when using visible - light images to enhance the texture details of infrared images, resulting in a decline in image quality.
- The paper proposes a new framework, target - oriented domain - adapted SRGAN (DASRGAN), which finely optimizes texture details through the texture - oriented adaptation (TOA) module to improve image quality.

2. **Noise suppression**:
- When using visible - light images for domain adaptation, how to effectively suppress the transfer of noise is a challenge.
- DASRGAN introduces a noise - oriented adaptation (NOA) module, which reduces the noise in the generated image and improves the clarity and authenticity of the image through the noise - oriented adversarial loss function.

3. **Effectiveness of domain adaptation**:
- The scarcity of infrared image samples limits the training effect of the model. The paper uses abundant visible - light image data for domain adaptation to improve the performance of infrared image super - resolution.
- DASRGAN effectively solves the feature space alignment problem in domain adaptation through a two - stage alternating training strategy, improving the generalization ability of the model.

### Main contributions

1. **Texture - oriented adaptation (TOA)**:
- A special discriminator (Dstrans) is introduced, which contains a Sobel prior extraction branch for optimizing texture details in adversarial training.
- A new texture - oriented prior adversarial loss function (Ltrans) is proposed to guide the generation network (G) to perform precise alignment on texture patterns.

2. **Noise - oriented adaptation (NOA)**:
- A new noise - oriented adversarial loss function (LGn) is designed, which reduces the transfer of noise by quantifying the perceptual differences between the predicted image and the artificially generated noise patterns.
- The pre - trained VGG network is used to extract hierarchical features and calculate losses at different layers to guide the network to generate images different from natural noise patterns.

3. **Experimental verification**:
- Through experiments on multiple benchmark datasets (M3FD, CVC), the superior performance of DASRGAN in the infrared image super - resolution task is verified.
- The experimental results show that DASRGAN is significantly superior to existing methods in terms of PSNR, MSE, and SSIM, especially performing well at different scales.

In conclusion, through the innovative DASRGAN framework, the paper effectively solves the texture enhancement and noise suppression problems in infrared image super - resolution, providing new directions and solutions for research in this field.