RSHazeDiff: A Unified Fourier-aware Diffusion Model for Remote Sensing Image Dehazing

Jiamei Xiong,Xuefeng Yan,Yongzhen Wang,Wei Zhao,Xiao-Ping Zhang,Mingqiang Wei

2024-05-15

Abstract:Haze severely degrades the visual quality of remote sensing images and hampers the performance of automotive navigation, intelligent monitoring, and urban management. The emerging denoising diffusion probabilistic model (DDPM) exhibits the significant potential for dense haze removal with its strong generation ability. Since remote sensing images contain extensive small-scale texture structures, it is important to effectively restore image details from hazy images. However, current wisdom of DDPM fails to preserve image details and color fidelity well, limiting its dehazing capacity for remote sensing images. In this paper, we propose a novel unified Fourier-aware diffusion model for remote sensing image dehazing, termed RSHazeDiff. From a new perspective, RSHazeDiff explores the conditional DDPM to improve image quality in dense hazy scenarios, and it makes three key contributions. First, RSHazeDiff refines the training phase of diffusion process by performing noise estimation and reconstruction constraints in a coarse-to-fine fashion. Thus, it remedies the unpleasing results caused by the simple noise estimation constraint in DDPM. Second, by taking the frequency information as important prior knowledge during iterative sampling steps, RSHazeDiff can preserve more texture details and color fidelity in dehazed images. Third, we design a global compensated learning module to utilize the Fourier transform to capture the global dependency features of input images, which can effectively mitigate the effects of boundary artifacts when processing fixed-size patches. Experiments on both synthetic and real-world benchmarks validate the favorable performance of RSHazeDiff over multiple state-of-the-art methods. Source code will be released at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is some limitations of the existing diffusion - model - based methods in the remote - sensing image dehazing task. Specifically, these problems include:

1. **Simple Noise Estimation Constraint**: The existing Denoising Diffusion Probability Model (DDPM) only uses simple noise estimation constraints to optimize the network during the training process, which may lead to unsatisfactory content and unnecessary impurities in the recovered image.
2. **Structural and Statistical Information in the Forward Iteration Process Not Fully Utilized**: DDPM fails to utilize the potential structural and statistical information in the forward iteration process, limiting its better sampling ability. Especially in the early sampling steps, it is difficult to supplement structural and color features for images in heavy - fog scenes.
3. **Patch - Based Processing Method Leads to Obvious Block - like Artifacts**: Although the patch - based DDPM can handle images of any size and accelerate sampling, this method ignores the global semantic information of all patches, resulting in obvious block - like artifacts in the final result.

To solve the above problems, the paper proposes a new Unified Fourier - aware Diffusion Model (RSHazeDiff) specifically for remote - sensing image dehazing. The main contributions of RSHazeDiff are as follows:

1. **Phased Training Strategy (PTS)**: By first training the diffusion model with noise estimation constraints and then further optimizing the model with reconstruction constraints, the unsatisfactory results caused by simple noise estimation constraints are improved.
2. **Fourier - aware Iterative Refinement Module (FIR)**: In each sampling step, FIR extracts style / semantic information from the magnitude / phase components in the Fourier domain to obtain more refined sampling results and better preserve the texture details and color fidelity of the recovered image.
3. **Global Compensated Learning Module (GCL)**: By capturing the global dependent features of the input image, it effectively eliminates the obvious block - like artifacts caused by the patch - based image processing scheme and ensures the semantic consistency of all patches.

Through these innovations, RSHazeDiff shows performance superior to that of multiple existing methods in synthetic and real - world benchmark tests, especially in heavy - fog scenes.