DMFourLLIE: Dual-Stage and Multi-Branch Fourier Network for Low-Light Image Enhancement

Tongshun Zhang,Pingping Liu,Ming Zhao,Haotian Lv
DOI: https://doi.org/10.1145/3664647.3681083
2024-12-01
Abstract:In the Fourier frequency domain, luminance information is primarily encoded in the amplitude component, while spatial structure information is significantly contained within the phase component. Existing low-light image enhancement techniques using Fourier transform have mainly focused on amplifying the amplitude component and simply replicating the phase component, an approach that often leads to color distortions and noise issues. In this paper, we propose a Dual-Stage Multi-Branch Fourier Low-Light Image Enhancement (DMFourLLIE) framework to address these limitations by emphasizing the phase component's role in preserving image structure and detail. The first stage integrates structural information from infrared images to enhance the phase component and employs a luminance-attention mechanism in the luminance-chrominance color space to precisely control amplitude enhancement. The second stage combines multi-scale and Fourier convolutional branches for robust image reconstruction, effectively recovering spatial structures and textures. This dual-branch joint optimization process ensures that complex image information is retained, overcoming the limitations of previous methods that neglected the interplay between amplitude and phase. Extensive experiments across multiple datasets demonstrate that DMFourLLIE outperforms current state-of-the-art methods in low-light image enhancement. Our code is available at <a class="link-external link-https" href="https://github.com/bywlzts/DMFourLLIE" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the technical challenges of image enhancement under low - light conditions. Specifically, when using the Fourier transform, existing low - light image enhancement techniques mainly focus on amplifying the magnitude component and simply copying the phase component, which often leads to color distortion and noise problems. The paper proposes a framework named DMFourLLIE (Dual - Stage Multi - Branch Fourier Low - Light Image Enhancement), aiming to overcome these limitations by emphasizing the role of the phase component in maintaining image structure and details. ### Main Problems and Solutions 1. **Limitations of Existing Methods**: - **Amplification of Magnitude Component**: Existing methods mainly enhance image brightness by amplifying the magnitude component, but this approach is prone to cause color distortion and increased noise. - **Insufficient Processing of Phase Component**: The phase component contains important spatial structure information, but existing methods usually simply copy the phase component and fail to fully utilize its information. 2. **Innovations of the DMFourLLIE Framework**: - **Two - stage Design**: - **First Stage**: Combine the structural information of infrared images to enhance the phase component, and use the luminance attention mechanism in the luminance - chrominance color space to precisely control the magnitude enhancement. - **Second Stage**: Combine multi - scale convolutional branches and Fourier convolutional branches for robust image reconstruction, effectively restoring spatial structure and texture. - **Cross - modal Enhancement**: - **Infrared Branch**: Use a pre - trained RGB - to - infrared image translation model to generate infrared images and extract the phase component to enhance the image structure. - **Brightness Branch**: Guide the magnitude enhancement in different regions through the luminance attention map to avoid over - exposure problems. - **Loss Function**: Comprehensively consider perceptual information and regional differences. The loss function consists of four parts: the L2 loss of the preliminary enhancement result, the L2 loss of the final output, the perceptual loss, and the L2 loss of the luminance attention map. ### Experimental Verification - **Data Set**: The paper conducted experiments on several commonly used data sets, including LOL - v2 - Real, LOL - v2 - synthesis, and LSRW - Huawei. - **Performance Comparison**: The experimental results show that DMFourLLIE outperforms the current state - of - the - art methods in terms of PSNR, SSIM, and LPIPS and other indicators. - **Ablation Experiment**: Ablation experiments were carried out by removing specific components to verify the effectiveness of each module. ### Conclusion DMFourLLIE effectively solves the color distortion and noise problems in existing low - light image enhancement methods by simultaneously enhancing the magnitude and phase components in the Fourier domain and combining multi - scale convolution and Fourier convolution techniques, significantly improving the image quality and visual effect.