Abstract:Face image-sketch synthesis is widely applied in law enforcement and digital entertainment fields. Despite the extensive progression in face image-sketch synthesis, there are few methods focusing on generating a color face image from a sketch. The existing methods pay less attention to learning the illumination or highlight distribution on the face region. However, the illumination is the key factor that makes the generated color face image looks vivid and realistic. Moreover, existing methods tend to employ some image preprocessing technologies and facial region patching approaches to generate high-quality face images, which results in the high complexity and memory consumption in practice. In this paper, we propose a novel end-to-end generative adversarial fusion model, called GAF, which fuses two U-Net generators and a discriminator by jointly learning the content and adversarial loss functions. In particular, we propose a parametric tanh activation function to learn and control illumination highlight distribution over faces, which is integrated between the two U-Net generators by an illumination distribution layer. Additionally, we fuse the attention mechanism into the second U-Net generator of GAF to keep the identity consistency and refine the generated facial details. The qualitative and quantitative experiments on the public benchmark datasets show that the proposed GAF has better performance than existing image-sketch synthesis methods in synthesized face image quality (FSIM) and face recognition accuracy (NLDA). Meanwhile, the good generalization ability of GAF has also been verified. To further demonstrate the reliability and authenticity of face images generated using GAF, we use the generated face image to attack the well-known face recognition system. The result shows that the face images generated by GAF can maintain identity consistency and well maintain everyone’s unique facial characteristics, which can be further used in the benchmark of facial spoofing. Moreover, the experiments are implemented to verify the effectiveness and rationality of the proposed parametric tanh activation function and attention mechanism in GAF.

Joint Sketch-Attribute Learning for Fine-Grained Face Synthesis.

Attribute-Guided Sketch Generation

Fine-grained Semantic Constraint in Image Synthesis

Fine-Granularity Face Sketch Synthesis

Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs

Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network

Facial Synthesis from Visual Attributes via Sketch using Multi-Scale Generators

HCGAN: hierarchical contrast generative adversarial network for unpaired sketch face synthesis

Toward Realistic Face Photo–Sketch Synthesis via Composition-Aided GANs

Composition-Aided Face Photo-Sketch Synthesis.

Recognizing Facial Sketches by Generating Photorealistic Faces Guided by Descriptive Attributes

Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning

Quality Guided Sketch-to-Photo Image Synthesis

Robust Face Sketch Synthesis Via Generative Adversarial Fusion of Priors and Parametric Sigmoid

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

Face photo-line drawings synthesis based on local extraction preserving generative adversarial networks

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

Recognizing Minimal Facial Sketch by Generating Photorealistic Faces with the Guidance of Descriptive Attributes

Multi-Level Cycle-Consistent Adversarial Networks with Attention Mechanism for Face Sketch-Photo Synthesis

Face image-sketch synthesis via generative adversarial fusion