Semantically-guided low-light image enhancement

Junyi Xie,Hao Bian,Yuanhang Wu,Yu Zhao,Linmin Shan,Shijie Hao

DOI: https://doi.org/10.1016/j.patrec.2020.07.041

IF: 4.757

2020-10-01

Pattern Recognition Letters

Abstract:<p>Recently, extensive research efforts have been made on low-light image enhancement. Many novel models have been proposed, such as the ones based on the Retinex theory, multiple exposure fusion, and deep neural networks. However, current models do not directly consider the semantic information in the modeling process. As a result, they tend to introduce more artifacts, such as boosted noises and unnatural visual appearances. To address this issue, we propose a fusion-based low-light enhancement model that explicitly harnesses the scene semantics into the enhancement process. In constructing the fusion map, the image regions with a specific semantic category is firstly extracted via semantic segmentation. Then, they are further combined and refined jointly with an illumination-aware map estimated from the scene illumination. Guided by the semantic information, our model is able to intentionally enhance a part of the dark regions, which therefore generates enhanced results with more natural appearances and less artifacts. In experiments, we first validate our model with some empirical studies, including parameter sensitivity and segmentation error tolerance. Then we compare our model with several state-of-the-art low-light enhancement methods, which further shows the effectiveness and advantage of our model.</p>

computer science, artificial intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to improve image quality, reduce artifacts generated during the enhancement process, and maintain visual naturalness by introducing scene semantic information in the process of enhancing images taken under low - light conditions. Specifically, although current low - light image enhancement models can improve the visibility of images, they are prone to introduce artifacts such as noise and halos during the enhancement process, resulting in unnatural visual effects. In addition, these models are usually unable to distinguish different semantic regions, causing some regions that should remain relatively dark (such as the night sky) to be over - enhanced.

To solve these problems, the author proposes a fusion - based low - light image enhancement model, which explicitly utilizes scene semantic information during the enhancement process. By constructing a fusion graph that takes into account illumination, local structure, and scene semantics simultaneously, this model can selectively enhance specific regions in the image, thereby generating more natural enhancement results with fewer artifacts.