Abstract:Images captured in low-light conditions often suffer from bad visibility, e.g., low contrast, lost details, and color distortion, and image enhancement methods can be used to improve the image quality. Previous methods have generally obtained a smooth illumination map to enhance the image but have ignored details, leading to inaccurate illumination estimations. To solve this problem, we propose a multi-scale attention retinex network (MARN) for low-light image enhancement, which learns an image-to-illumination mapping to obtain a detailed inverse illumination map inspired by retinex theory. In order to introduce more image priors, we introduce a novel illuminance-attention map to guide the model to characterize varying-lighting areas, which we combine with the low-light image as the model input. MARN consists of a multi-scale attention module and a feature fusion module; the former extracts multi-resolution features with attention-based feature aggregation, while the latter further merges the output features of the previous module with the input. To achieve better visibility, we formulate a novel loss function to synthetically measure the illumination, detail, and colorfulness of the image. Extensive experiments are performed on several benchmark datasets. The results demonstrate that our method outperforms other state-of-the-art methods according to both objective and subjective metrics.
computer science, information systems,telecommunications,engineering, electrical & electronic
What problem does this paper attempt to address?
The paper attempts to address the issue of poor image quality captured under low-light conditions. Specifically, these images often suffer from low contrast, loss of details, and color distortion, which affect the visibility of the images and the effectiveness of subsequent processing (such as object detection and segmentation). To solve these problems, the paper proposes a Multi-Scale Attention Retinex Network (MARN) for low-light image enhancement. MARN learns the mapping from the image to the illumination to obtain a detailed inverse illumination map and combines multi-scale attention mechanisms and feature fusion modules to improve the visibility, details, and color restoration of the images.
### Main Contributions of the Paper:
1. **Proposed a new method**: MARN, to obtain a detailed inverse illumination map to enhance low-light images. To introduce more image prior information, the paper introduces a new illumination attention map that guides the model to represent different illumination regions and combines it with low-light images as network input to estimate a more accurate illumination distribution.
2. **Designed a multi-scale attention module**: This module can extract features at different scales and integrate them through attention-based feature aggregation, improving the network's generalization ability.
3. **Introduced a comprehensive loss function**: This loss function covers structural similarity, detail, and color recovery, including reference and non-reference components, to improve the visibility of the results.
4. **Conducted extensive experiments**: Experiments were conducted on multiple benchmark datasets, and the results showed that the proposed method outperformed existing state-of-the-art methods in both objective and subjective metrics. Additionally, ablation studies were conducted to demonstrate the effectiveness of the proposed structure.
### Related Work:
- **Traditional methods**: Include histogram equalization and methods based on retinex theory. Histogram equalization methods improve contrast by expanding the image histogram but may lead to over-enhancement or under-enhancement. Retinex theory methods decompose the image into reflectance and illumination components and improve image quality by enhancing the illumination component.
- **Learning-based methods**: In recent years, deep learning methods have made significant progress in low-light image enhancement. Some methods directly learn from low-light images to normal-light images in an end-to-end manner, while others learn illumination and reflectance components separately through multiple CNNs. There are also unsupervised methods that use Generative Adversarial Networks (GANs) or non-reference loss functions to estimate normal-light images.
### Method Overview:
1. **Retinex theory enhancement model**: Based on retinex theory, the image is decomposed into reflectance and illumination components, and the image is enhanced by estimating the inverse illumination map.
2. **Illumination attention map**: Introduces an illumination attention map to provide more prior information, allowing the network to focus more on underexposed areas and avoid over-enhancing normally exposed areas.
3. **Network structure**: MARN includes multi-scale attention modules and feature fusion modules. The multi-scale attention module extracts multi-resolution features through parallel branches and aggregates features through attention mechanisms. The feature fusion module further integrates the output features to generate the final inverse illumination map.
4. **Loss function**: Includes reconstruction loss, structural similarity loss, color loss, and detail loss to comprehensively evaluate the quality of the image.
### Experimental Setup:
- **Dataset**: The MIT-Adobe FiveK dataset is used as the training set, containing 5000 original images. Each image is retouched by different experts using Adobe Lightroom software, and the output of Expert E is chosen as the ground truth.
- **Implementation details**: The model is implemented using the PyTorch framework, with the Adam optimizer, an initial learning rate of 1e-3, and a decay of 0.5 every 150 epochs. All experiments are conducted on two NVIDIA GeForce GTX1080TI GPUs.
### Experimental Results:
- **Ablation studies**: Verified the effectiveness of the illumination attention map, multi-scale attention module, and different loss functions.
- **Evaluation on benchmark datasets**: Compared with various state-of-the-art low-light image enhancement methods, the results show that MARN performs excellently in both visual effects and objective evaluation metrics.
In summary, the paper proposes a new Multi-Scale Attention Retinex Network (MARN) that effectively addresses multiple challenges in low-light image enhancement, improving the visibility, details, and color restoration of images.