Abstract:The rapid development of social network and online multimedia technology makes it possible to address traditional image and video enhancement problems, with the aid of online similar reference data. In this paper, we tackle the problem of super-resolution (SR) in this way, specifically aiming to handle the “one-to-many” problem between the image patches of low resolution (LR) and high resolution (HR). We propose a manifold localized deep external compensation (MALDEC) network to additionally utilize reference images, i. e., retrieved similar images in cloud database and reference HR frame in a video, to provide an accurate localization and mapping to the HR manifold, and compensate the lost high-frequency details. The proposed network employs a three-step recovery: 1) internal structure inference, which uses the LR image itself and the internally inferred high frequency information to preserve main structure of the HR image; 2) manifold localization, which localizes the HR manifold and constructs the correspondence between the internal inferred image and the external images; and 3) external compensation, which introduces the external references of retrieved similar patches based on manifold localization information to reconstruct the high-frequency details. The learnable components of MALDEC, internal structure inference, and external compensation, are trained jointly to make a good tradeoff between these two terms for an optimal SR result. Finally, the proposed method is examined under three tasks: cloud-based image SR, multi-pose face reconstruction, and reference frame-guided video SR. Extensive experiments demonstrate the superiority of our method than the state-of-the-art SR methods in both objective and subjective evaluations, and our method offers new state-of-the-art performance.

Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution

Face Super-Resolution Via Bilayer Contextual Representation

Cross-resolution Face Recognition with Pose Variations Via Multilayer Locality-Constrained Structural Orthogonal Procrustes Regression.

Bilateral Structure Based Matrix Regression Classification for Face Recognition

Robust Low-Resolution Face Recognition Via Low-Rank Representation and Locality-Constrained Regression

The face image super-resolution algorithm based on combined representation learning

Locality-Constrained Discriminative Matrix Regression for Robust Face Identification

Learning Locality-Constrained Collaborative Representation for Face Recognition

Locality-Constrained Structral Orthogonal Procrustes Regression For Low-Resolution Face Recognition With Pose Variations

Self-adjusting multilayer nonlinear coupled mapping for low-resolution face recognition

Two-Stage Multi-Scale Resolution-Adaptive Network for Low-Resolution Face Recognition

Robust Face Image Super-Resolution via Joint Learning of Subdivided Contextual Model

Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition

Robust Face Hallucination Via Locality-Constrained Nuclear Norm Regularized Regression

Modeling and Optimizing of the Multi-Layer Nearest Neighbor Network for Face Image Super-Resolution

Robust Face Recognition Via Multi-Scale Patch-Based Matrix Regression

Robust Face Super-Resolution Via Position Relation Model Based on Global Face Context.

Multi-scale Patch Based Representation Feature Learning for Low-Resolution Face Recognition

Efficient Face Alignment via Locality-constrained Representation for Robust Recognition

Reference-Guided Deep Super-Resolution Via Manifold Localized External Compensation

Local Sparse Representation Projections for Face Recognition.