Zero-Shot Low-Light Image Enhancement via Joint Frequency Domain Priors Guided Diffusion

Jinhong He,Shivakumara Palaiahnakote,Aoxiang Ning,Minglong Xue

2024-11-21

Abstract:Due to the singularity of real-world paired datasets and the complexity of low-light environments, this leads to supervised methods lacking a degree of scene generalisation. Meanwhile, limited by poor lighting and content guidance, existing zero-shot methods cannot handle unknown severe degradation well. To address this problem, we will propose a new zero-shot low-light enhancement method to compensate for the lack of light and structural information in the diffusion sampling process by effectively combining the wavelet and Fourier frequency domains to construct rich a priori information. The key to the inspiration comes from the similarity between the wavelet and Fourier frequency domains: both light and structure information are closely related to specific frequency domain regions, respectively. Therefore, by transferring the diffusion process to the wavelet low-frequency domain and combining the wavelet and Fourier frequency domains by continuously decomposing them in the inverse process, the constructed rich illumination prior is utilised to guide the image generation enhancement process. Sufficient experiments show that the framework is robust and effective in various scenarios. The code will be available at: \href{<a class="link-external link-https" href="https://github.com/hejh8/Joint-Wavelet-and-Fourier-priors-guided-diffusion" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/hejh8/Joint-Wavelet-and-Fourier-priors-guided-diffusion" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of image enhancement under low - light conditions, especially in the zero - shot scenario. Specifically, the existing methods have the following problems:

1. **Limitations of supervised methods**: Due to the singularity of real - world paired datasets and the complexity of low - light environments, supervised methods lack scene generalization ability.
2. **Deficiencies of zero - shot methods**: Existing zero - shot methods cannot handle unknown severe degradation situations well because of the limitations of illumination and content guidance.

To solve these problems, the authors propose a new zero - shot low - light image enhancement method. This method constructs rich prior information by effectively combining the wavelet domain and the Fourier frequency domain, thereby compensating for the lack of illumination and structural information in the diffusion sampling process.

### Main contributions

1. **Propose a zero - shot low - light image enhancement method**: By constructing rich prior information in the joint wavelet and Fourier frequency domains, it effectively compensates for the lack of illumination and structural information in the zero - shot diffusion enhancement process.
2. **Explore the effective combination of the wavelet and Fourier frequency domains**: To meet the degradation and enhancement requirements in different complex scenarios. At the same time, multi - modal text embedding is introduced to guide the positive enhancement direction.
3. **Extensive experiments verify the effectiveness and universality of this method**.

### Method overview

- **Diffusion model preliminaries**: Introduces the basic principles of the diffusion model, including the forward diffusion process and the reverse sampling process.
- **Joint wavelet and Fourier prior - guided sampling**: Decomposes the frequency - domain information of the input image through the discrete wavelet transform (DWT) and the fast Fourier transform (FFT), and combines this information to guide the sampling process.
- **Network optimization**: By embedding multi - modal text and supervising the enhancement process during the diffusion process, it ensures the consistency of visual perception and luminance distribution.

### Experimental results

- **Quantitative comparison**: Quantitative evaluations were carried out on multiple benchmark datasets, and the results show that this method achieved top - three performance on all real datasets, especially performing best among zero - shot methods.
- **Qualitative comparison**: Compared with existing methods, the images generated by this method have more realistic colors and details, and the visual effects are more in line with human perception.

In conclusion, this paper proposes an innovative zero - shot low - light image enhancement method. By combining the information in the wavelet and Fourier frequency domains, it significantly improves the effect and generalization ability of image enhancement.