Abstract:Understanding illumination and reducing the need for supervision pose a significant challenge in low-light enhancement. Current approaches are highly sensitive to data usage during training and illumination-specific hyper-parameters, limiting their ability to handle unseen scenarios. In this paper, we propose a new zero-reference low-light enhancement framework trainable solely with normal light images. To accomplish this, we devise an illumination-invariant prior inspired by the theory of physical light transfer. This prior serves as the bridge between normal and low-light images. Then, we develop a prior-to-image framework trained without low-light data. During testing, this framework is able to restore our illumination-invariant prior back to images, automatically achieving low-light enhancement. Within this framework, we leverage a pretrained generative diffusion model for model ability, introduce a bypass decoder to handle detail distortion, as well as offer a lightweight version for practicality. Extensive experiments demonstrate our framework's superiority in various scenarios as well as good interpretability, robustness, and efficiency. Code is available on our project homepage:
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the challenge of image enhancement under low - light conditions, especially how to achieve effective low - light enhancement in the absence of supervision information (i.e., unpaired or non - paired data). Specifically, current methods highly depend on data usage and hyper - parameters under specific lighting conditions during the training process, which limits their ability to handle unseen scenes.
To solve these problems, the authors propose a new zero - reference low - light enhancement framework, which is trained only with normal - light images. By introducing a lighting - invariant prior (physical quadruple prior) based on the physical lighting - transfer theory, this framework can restore these prior features to the image during testing, thereby automatically achieving low - light enhancement. The following are the main contributions of the paper:
1. **Propose a zero - reference low - light enhancement model**: This model uses the lighting - invariant prior as an intermediary between different lighting conditions and can perform well in various low - light scenes without relying on any specific low - light data.
2. **Establish the Physical Quadruple Prior**: This is a novel learnable lighting - invariant prior derived from the lighting - transfer theory, which can capture the essence of imaging under different lighting conditions and make low - light enhancement independent of reference samples or artificially set hyper - parameters.
3. **Develop an effective prior - to - image mapping system**: By using the prior as a condition to control the pre - trained large - scale generative diffusion model, the problem of detail distortion is solved, and it is shown that this model can be simplified to a lightweight version to adapt to practical applications.
### Formula Summary
- **Energy formula in Kubelka - Munk theory**:
\[
E(\lambda, x)=e(\lambda, x)\left(\frac{1 - i(x)}{2R_{\infty}(\lambda, x)}+i(x)\right)
\]
where \(e(\lambda, x)\) represents the spectrum of the light source, \(i(x)\) represents specular reflection, and \(R_{\infty}(\lambda, x)\) represents the reflectivity of the material.
- **Simplified energy formula** (when the object is matte):
\[
E(\lambda, x)=e(\lambda, x)R_{\infty}(\lambda, x)
\]
- **Light - invariant feature H**:
\[
H = \arctan\left(\frac{E_{\lambda}}{E_{\lambda\lambda}}\right)
\]
- **Light - invariant feature C**:
\[
C=\log\left(\frac{(E_{\lambda})^2+(E_{\lambda\lambda})^2}{E(\lambda, x)^2}\right)=\log\left(\frac{(R_{\infty\lambda})^2+(R_{\infty\lambda\lambda})^2}{R_{\infty}(\lambda, x)^2}\right)
\]
- **Light - invariant feature W**:
\[
W = \tan\left(\left|\frac{\partial E(\lambda, x)}{\partial x_1}\right|/E(\lambda, x)\right)=\tan\left(\left|\frac{\partial R_{\infty}(\lambda, x)}{\partial x_1}\right|/R_{\infty}(\lambda, x)\right)
\]
- **RGB channel order O**:
\[
O(x, y)=[O_R(x, y), O_G(x, y), O_