Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement

Yibing Song,Jiawei Zhang,Lijun Gong,Shengfeng He,Linchao Bao,Jinshan Pan,Qingxiong Yang,Ming-Hsuan Yang

DOI: https://doi.org/10.48550/arXiv.1811.09019

2018-11-22

Abstract:We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image. However, the CNN based method cannot handle image details well. We further develop a novel exemplar-based detail enhancement algorithm via facial component matching. Extensive experiments show that the proposed method outperforms the state-of-the-art algorithms both quantitatively and qualitatively.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to restore clear high - resolution (HR) face images from blurry low - resolution (LR) face images. This problem is challenging because it is necessary to handle super - resolution (SR) and deblurring simultaneously. Existing algorithms perform poorly when dealing with face images because low - resolution face images lack sufficient texture details, which are especially important for deblurring. In addition, when the input image contains severe motion blur, the effectiveness of existing methods is also not satisfactory. This paper proposes an effective method to restore high - quality face images by leveraging specific knowledge in the face domain.

Specifically, the author proposes a deep convolutional neural network (CNN) guided by facial components to restore rough face images, and further develops an example - based detail enhancement algorithm to enhance image details through facial component matching. Experimental results show that the proposed method outperforms the existing state - of - the - art algorithms both quantitatively and qualitatively.