Abstract:Face sketch synthesis has a wide range of applications in both digital entertainment and law enforcement. State-of-the-art examplar-based methods typically exploit a Probabilistic Graphical Model (PGM) to represent the joint probability distribution over all of the patches selected from a set of training data. However, these methods suffer from two main shortcomings: (1) most of these methods capture the evidence between patches in pixel-level, which lead to inaccurate parameter estimation under bad environment conditions such as light variations and clutter backgrounds; (2) the assumption that a photo patch and its corresponding sketch patch share similar geometric manifold structure is not rigorous. It has shown that deep convolutional neural network (CNN) has outstanding performance in learning to extract high-level feature representation. Therefore, we extract uniform deep patch representations of test photo patches and training sketch patches from a specially designed CNN model to replace pixel intensity, and directly match between them, which can help select better candidate patches from training data as well as improve parameter learning process. In this way, we investigate a novel face sketch synthesis method called DPGM that combines generative PGM and discriminative deep patch representation, which can jointly model the distribution over the parameters for deep patch representation and the distribution over the parameters for sketch patch reconstruction. Then, we apply an alternating iterative optimization strategy to simultaneously optimize two kinds of parameters. Therefore, both the representation capability of deep patch representation and the reconstruction ability of sketch patches can be boosted. Eventually, high quality reconstructed sketches which is robust against light variations and clutter backgrounds can be obtained. Extensive experiments on several benchmark datasets demonstrate that our method can achieve superior performance than other state-of-the-art methods, especially under the case of bad light conditions or clutter backgrounds.

Towards Criminal Sketching with Generative Adversarial Network

Attribute-Guided Sketch Generation

Evaluating the performance of face sketch generation using generative adversarial networks

Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network

Recognizing Facial Sketches by Generating Photorealistic Faces Guided by Descriptive Attributes

Quality Guided Sketch-to-Photo Image Synthesis

Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning

SmartPaint: a Co-Creative Drawing System Based on Generative Adversarial Networks

Face photo-line drawings synthesis based on local extraction preserving generative adversarial networks

End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning

HCGAN: hierarchical contrast generative adversarial network for unpaired sketch face synthesis

Face photo-sketch portraits transformation via generation pipeline

Toward Realistic Face Photo–Sketch Synthesis via Composition-Aided GANs

Composition-Aided Face Photo-Sketch Synthesis.

Staged Sketch-to-Image Synthesis via Semi-supervised Generative Adversarial Networks

Robust Face Sketch Synthesis Via Generative Adversarial Fusion of Priors and Parametric Sigmoid

Learning Deep Patch representation for Probabilistic Graphical Model-Based Face Sketch Synthesis

Learning to Sketch Human Facial Portraits Using Personal Styles by Case-Based Reasoning

Fine-Granularity Face Sketch Synthesis

Recognizing Minimal Facial Sketch by Generating Photorealistic Faces with the Guidance of Descriptive Attributes