Reference-guided face inpainting with reference attention network

Jiazuo Yu,Kai Li,Jinjia Peng
DOI: https://doi.org/10.1007/s00521-022-06961-8
2022-02-03
Neural Computing and Applications
Abstract:Face inpainting is a significant problem encountered in many image restoration tasks, in which various methods based on deep learning are explored. Existing methods cannot restore enough structure details as the masked input only provides limited information. In this paper, a novel reference-guided face inpainting method is proposed to generate inpainting results more similar to people themselves, which restores the missing pixels by referring to a reference image besides an original masked image. Concretely, another reference image with the same identity as the masked input is utilized as a conditional input to constrain the generated coarse result of the first inpainting stage. Furthermore, a reference attention module is designed to restore more textural details by computing the similarity between the pixels of the coarse result and the reference image. The similarity is further represented by the similarity maps, which are deconvolved to reconstruct the pixels of the missing regions. Extensive experimental results on CelebA datasets and LFW datasets demonstrate that our proposed method can generate an image with more similar features to people themselves and achieves superior performance to the state-of-the-art methods quantitatively and qualitatively.
computer science, artificial intelligence
What problem does this paper attempt to address?