Abstract:Biphasic face photo-sketch synthesis has significant practical value in wide-ranging fields such as digital entertainment and law enforcement. Previous approaches directly generate the photo-sketch in a global view, they always suffer from the low quality of sketches and complex photo variations, leading to unnatural and low-fidelity results. In this paper, we propose a novel Semantic-Driven Generative Adversarial Network to address the above issues, cooperating with Graph Representation Learning. Considering that human faces have distinct spatial structures, we first inject class-wise semantic layouts into the generator to provide style-based spatial information for synthesized face photos and sketches. Additionally, to enhance the authenticity of details in generated faces, we construct two types of representational graphs via semantic parsing maps upon input faces, dubbed the IntrA-class Semantic Graph (IASG) and the InteR-class Structure Graph (IRSG). Specifically, the IASG effectively models the intra-class semantic correlations of each facial semantic component, thus producing realistic facial details. To preserve the generated faces being more structure-coordinated, the IRSG models inter-class structural relations among every facial component by graph representation learning. To further enhance the perceptual quality of synthesized images, we present a biphasic interactive cycle training strategy by fully taking advantage of the multi-level feature consistency between the photo and sketch. Extensive experiments demonstrate that our method outperforms the state-of-the-art competitors on the CUFS and CUFSF datasets.

Semi-parametric Image Synthesis

Fine-grained Semantic Constraint in Image Synthesis

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Semantic Image Synthesis with Spatially-Adaptive Normalization

Diverse Semantic Image Synthesis via Probability Distribution Modeling

Semantic Probability Distribution Modeling for Diverse Semantic Image Synthesis

Label-free Neural Semantic Image Synthesis

Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning

Realistic Image Synthesis with Configurable 3D Scene Layouts

Semantic Image Synthesis with Unconditional Generator

Semantic RGB-D Image Synthesis

Semantic Image Synthesis via Class-Adaptive Cross-Attention

Generative View Synthesis: From Single-view Semantics to Novel-view Images

Unpaired semantic neural person image synthesis

Semantic Image Synthesis via Adversarial Learning

Unsupervised Scene Sketch to Photo Synthesis

PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis

On the Diversity of Conditional Image Synthesis with Semantic Layouts.

Staged Sketch-to-Image Synthesis via Semi-supervised Generative Adversarial Networks

Learning semantic priors for texture-realistic sketch-to-image synthesis

Semantic Image Synthesis Via Diffusion Models