ILSR-Diff: joint face illumination normalization and super-resolution via diffusion models

Wei Wang,Minghao Mu,Yan Tian,Yaocong Hu,Xiaobo Lu
DOI: https://doi.org/10.1007/s00530-024-01515-5
IF: 3.9
2024-10-05
Multimedia Systems
Abstract:The existing diffusion models (DMs) have shown impressive performance in face super-resolution tasks under normal illumination conditions. However, when applied to low-resolution (LR) facial images captured under non-uniform illumination (NI) conditions, the performance of DMs significantly deteriorates due to the lack of sufficient illumination information constraints. To address this challenge, we present ILSR-Diff, a novel illumination-constrained DM, designed to restore authentic high-resolution (HR) facial images while compensating for NI conditions. ILSR-Diff comprises three key components: (1) The illumination conditional constraint module extracts global and spatial illumination constraints from conditional facial images and seamlessly embeds them into the denoising network, which effectively addresses facial distortion caused by complex facial illumination variations in LR faces. (2) The noise guider harnesses facial priors by blending them with Gaussian noise to guide the initial noise sampling, which provides superior sampling initialization and prior knowledge for the denoising process to enhance the efficiency and precision of reconstruction. (3) The contrastive diffusion loss function utilizes high-level semantic features of high-quality faces to supervise the training of the denoising network, thereby enhancing the recovery details of facial features. Extensive experiments demonstrate that ILSR-Diff achieves authentic HR facial images under NI conditions and outperforms other state-of-the-art methods both qualitatively and quantitatively. Compared with SR3, our method achieves improvements of 2.2 dB and 2.8 dB in PSNR on the Multi-PIE and Extended-YaleB datasets, respectively.
computer science, information systems, theory & methods
What problem does this paper attempt to address?