DestripeCycleGAN: Stripe Simulation CycleGAN for Unsupervised Infrared Image Destriping

Shiqi Yang,Hanlin Qin,Shuai Yuan,Xiang Yan,Hossein Rahmani
2024-02-14
Abstract:CycleGAN has been proven to be an advanced approach for unsupervised image restoration. This framework consists of two generators: a denoising one for inference and an auxiliary one for modeling noise to fulfill cycle-consistency constraints. However, when applied to the infrared destriping task, it becomes challenging for the vanilla auxiliary generator to consistently produce vertical noise under unsupervised constraints. This poses a threat to the effectiveness of the cycle-consistency loss, leading to stripe noise residual in the denoised image. To address the above issue, we present a novel framework for single-frame infrared image destriping, named DestripeCycleGAN. In this model, the conventional auxiliary generator is replaced with a priori stripe generation model (SGM) to introduce vertical stripe noise in the clean data, and the gradient map is employed to re-establish cycle-consistency. Meanwhile, a Haar wavelet background guidance module (HBGM) has been designed to minimize the divergence of background details between the different domains. To preserve vertical edges, a multi-level wavelet U-Net (MWUNet) is proposed as the denoising generator, which utilizes the Haar wavelet transform as the sampler to decline directional information loss. Moreover, it incorporates the group fusion block (GFB) into skip connections to fuse the multi-scale features and build the context of long-distance dependencies. Extensive experiments on real and synthetic data demonstrate that our DestripeCycleGAN surpasses the state-of-the-art methods in terms of visual quality and quantitative evaluation. Our code will be made public at
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to address the issue of infrared image destriping. Specifically, the researchers propose a new framework called **DestripeCycleGAN** for removing vertical stripe noise in infrared images in an unsupervised manner. #### Core Issues: 1. **Limitations of traditional methods**: Traditional destriping methods often blur background details while removing stripe noise, leading to the loss of vertical edges. 2. **Shortcomings of existing deep learning methods**: Existing supervised learning-based methods require paired clean and striped image datasets, which are difficult to obtain in real-world scenarios. While CycleGAN-based methods can perform unsupervised learning, they are inadequate in handling directional stripe noise. #### Proposed New Method: - **Stripe Generation Model (SGM)**: Used to generate directional stripe noise, ensuring the directional characteristics of the noise. - **Haar Wavelet Background Guidance Module (HBGM)**: Used to maintain background consistency and eliminate the impact of vertical stripes. - **Multi-level Wavelet U-Net (MWUNet)**: Serves as the denoising generator, utilizing wavelet transform to reduce feature loss and integrate multi-scale information. With these improvements, DestripeCycleGAN can more effectively remove stripe noise in infrared images while preserving background details and vertical edges. Experimental results show that this method outperforms existing destriping methods across various datasets.