Spatial-frequency Dual-Domain Feature Fusion Network for Low-Light Remote Sensing Image Enhancement

Zishu Yao,Guodong Fan,Jinfu Fan,Min Gan,C.L. Philip Chen

DOI: https://doi.org/10.1109/TGRS.2024.3434416

2024-09-06

Abstract:Low-light remote sensing images generally feature high resolution and high spatial complexity, with continuously distributed surface features in space. This continuity in scenes leads to extensive long-range correlations in spatial domains within remote sensing images. Convolutional Neural Networks, which rely on local correlations for long-distance modeling, struggle to establish long-range correlations in such images. On the other hand, transformer-based methods that focus on global information face high computational complexities when processing high-resolution remote sensing images. From another perspective, Fourier transform can compute global information without introducing a large number of parameters, enabling the network to more efficiently capture the overall image structure and establish long-range correlations. Therefore, we propose a Dual-Domain Feature Fusion Network (DFFN) for low-light remote sensing image enhancement. Specifically, this challenging task of low-light enhancement is divided into two more manageable sub-tasks: the first phase learns amplitude information to restore image brightness, and the second phase learns phase information to refine details. To facilitate information exchange between the two phases, we designed an information fusion affine block that combines data from different phases and scales. Additionally, we have constructed two dark light remote sensing datasets to address the current lack of datasets in dark light remote sensing image enhancement. Extensive evaluations show that our method outperforms existing state-of-the-art methods. The code is available at <a class="link-external link-https" href="https://github.com/iijjlk/DFFN" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence,Image and Video Processing

What problem does this paper attempt to address?

The paper aims to address the problem of remote sensing image enhancement under low-light conditions. Specifically:

1. **Problem Background**: Remote sensing images captured at night or in other low-light environments often suffer from color degradation and detail loss, which can affect various downstream tasks that rely on these images, such as disaster recovery, wildlife monitoring, and environmental protection.

2. **Existing Method Shortcomings**:
- Convolutional Neural Networks (CNNs) struggle to establish long-range associations, especially in high-resolution images, where the local receptive field cannot capture global information.
- Transformer-based methods, while capable of handling long-range dependencies, have high computational complexity and are prone to overfitting on small datasets.

3. **Proposed Method**: To overcome these issues, the authors propose a Dual-Domain Feature Fusion Network (DFFN), which simplifies the entire process by decomposing the low-light image enhancement task into two sub-tasks:
- First Stage: Learning amplitude information to restore image brightness.
- Second Stage: Learning phase information to refine image details.

4. **Information Fusion Module**: An Information Fusion Affine Module (IFAM) is designed to facilitate the interaction of information at different scales and domains between the two-stage networks, enhancing the model's global contextual representation capability.

5. **Dataset Construction**: Due to the lack of available datasets, the authors constructed two large-scale low-light remote sensing image datasets, including a paired dataset iSAID-dark for training and a real dataset darkrs for testing.

In summary, this paper aims to effectively improve the enhancement of low-light remote sensing images by proposing a novel Dual-Domain Feature Fusion Network and corresponding information fusion mechanism.