Abstract:As a domain-specific super-resolution problem, facial image hallucination has enjoyed a series of breakthroughs thanks to the advances of deep convolutional neural networks. However, the direct migration of existing methods to video is still difficult to achieve good performance due to its lack of alignment and consistency modelling in temporal domain. Taking advantage of high inter-frame dependency in videos, we propose a self-enhanced convolutional network for facial video hallucination. It is implemented by making full usage of preceding super-resolved frames and a temporal window of adjacent low-resolution frames. Specifically, the algorithm first obtains the initial high-resolution inference of each frame by taking into consideration a sequence of consecutive low-resolution inputs through temporal consistency modelling. It further recurrently exploits the reconstructed results and intermediate features of a sequence of preceding frames to improve the initial super-resolution of the current frame by modelling the coherence of structural facial features across frames. Quantitative and qualitative evaluations demonstrate the superiority of the proposed algorithm against state-of-the-art methods. Moreover, our algorithm also achieves excellent performance in the task of general video super-resolution in a single-shot setting.

Multi-pose Face Hallucination Via Neighbor Embedding for Facial Components

FReeNet: Multi-Identity Face Reenactment

Face Hallucination Based on Neighbor Embedding Via Illumination Adaptation

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

Real-Time Audio-Guided Multi-Face Reenactment

Face Hallucination Based on Key Parts Enhancement

Self-Enhanced Convolutional Network for Facial Video Hallucination

Face Hallucination VIA Sparse Coding.

Robust Face Hallucination Via Locality-Constrained Nuclear Norm Regularized Regression

Hallucinating Faces: LPH Super-Resolution and Neighbor Reconstruction for Residue Compensation.

Video-based facial expression hallucination: a two- level hierarchical fusion approach

Contextual-Field Supported Iterative Representation for Face Hallucination.

Face Hallucination Using Deep Collaborative Representation for Local and Non-Local Patches

Face Hallucination Via Semi-Kernel Partial Least Squares

Identity-Preserving Face Hallucination via Deep Reinforcement Learning

A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model

Facial Expression Hallucination Through Eigen-Associative Learning.

A Simple Approach to Multiview Face Hallucination

An Improved Two-Step Approach to Hallucinating Faces

VidFace: A Full-Transformer Solver for Video FaceHallucination with Unaligned Tiny Snapshots