Abstract:Face sketch synthesis is a crucial issue in digital entertainment and law enforcement. It can bridge the considerable texture discrepancy between face photos and sketches. Most of the current face sketch synthesis approaches directly to learn the relationship between the photos and sketches, and it is very difficult for them to generate the individual specific features, which we call rare characteristics. In this paper, we propose a novel face sketch synthesis approach through residual learning. In contrast to traditional approaches, which aim to reconstruct a sketch image directly (i.e., learn the mapping relationship between the photo and sketch), we aim to predict the residual image by learning the mapping relationship between the photo and residual, i.e., the difference between the photo and sketch, given an observed photo. This technique will render optimizing the residual mapping easier than optimizing the original mapping and deriving rare characteristic information. We also introduce a joint dictionary learning algorithm by preserving the local geometry structure of a data space. Through the learned joint dictionary, we transform the face sketch synthesis from an image space to a new and compact space; the new and compact space is spanned by learned dictionary atoms, where the manifold assumption can be further guaranteed. Results show that the proposed method demonstrates an impressive performance in the face sketch synthesis task on three public face sketch datasets and various real-world photos. These results are derived by comparing the proposed method with several state-of-the-art techniques, including certain recently proposed deep learning-based approaches.

Coupled Dictionary Learning for the Detail-Enhanced Synthesis of 3-D Facial Expressions

Learning Individual-Specific Dictionaries With Fused Multiple Features For Face Recognition

3D Facial Expression Synthesis Based on Nonlinear Co-learning

Image-based facial sketch-to-photo synthesis via online coupled dictionary learning

High Resolution Acquisition, Learning And Transfer Of Dynamic 3-D Facial Expressions

Personalized 3-D facial expression synthesis based on landmark constraint.

A Unified Deep Model for Joint Facial Expression Recognition, Face Synthesis, and Face Alignment.

Joint sparse learning for 3-D facial expression generation.

Sparse coding for flexible, robust 3D facial-expression synthesis.

Structured Detail Enhancement for Cross-Modality Face Synthesis

Face Sketch-Photo Synthesis under Multi-dictionary Sparse Representation Framework

Enhanced Dictionary Pair Learning Sparse Representation Model For Facial Expression Classification

Facial Expression Synthesis Based on Semantic Dimensions

Facial Expression Synthesis and Recognition Using a Kernel-Based Factorization Model

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images

Image based Face Sketch-to-Photo via Online Coupled Dictionary Learning

Graph-Regularized Locality-Constrained Joint Dictionary and Residual Learning for Face Sketch Synthesis.

Synthesis of Combined Facial Expressions Using Anatomy-based Model

DNPM: A Neural Parametric Model for the Synthesis of Facial Geometric Details

Expressive 3D face synthesis by multi-space modeling

Facial Expression Synthesis Based on Facial Component Model.