Abstract:Recognizing the identity of a sketched face from a face photograph dataset is a critical yet challenging task in many applications, not least law enforcement and criminal investigations. An intelligent sketched face identification system would rely on automatic face sketch synthesis from photographs, thereby avoiding the cost of artists manually drawing sketches. However, conventional face sketch-photo synthesis methods tend to generate sketches that are consistent with the artists'drawing styles. Identity-specific information is often overlooked, leading to unsatisfactory identity verification and recognition performance. In this paper, we discuss the reasons why conventional methods fail to recover identity-specific information. Then, we propose a novel dual-transfer face sketch-photo synthesis framework composed of an inter-domain transfer process and an intra-domain transfer process. In the inter-domain transfer, a regressor of the test photograph with respect to the training photographs is learned and transferred to the sketch domain, ensuring the recovery of common facial structures during synthesis. In the intra-domain transfer, a mapping characterizing the relationship between photographs and sketches is learned and transferred across different identities, such that the loss of identity-specific information is suppressed during synthesis. The fusion of information recovered by the two processes is straightforward by virtue of an ad hoc information splitting strategy. We employ both linear and nonlinear formulations to instantiate the proposed framework. Experiments on The Chinese University of Hong Kong face sketch database demonstrate that compared to the current state-of-the-art the proposed framework produces more identifiable facial structures and yields higher face recognition performance in both the photo and sketch domains.

Sketch Transformer: Asymmetrical Disentanglement Learning from Dynamic Synthesis

SketchTrans: Disentangled Prototype Learning With Transformer for Sketch-Photo Recognition

Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation

A Sketch-Transformer Network for Face Photo-Sketch Synthesis

Sketch-Based 3D Shape Retrieval with Multi-View Fusion Transformer

Transformer-Based adversarial network for semi-supervised face sketch synthesis

Dual-Transfer Face Sketch-Photo Synthesis

Unsupervised Sketch-to-Photo Synthesis

Learning Dense Correspondences between Photos and Sketches

Photo-sketch Synthesis and Recognition Based on Subspace Learning

A modality separation approach for facial sketch synthesis

Face Sketch-Photo Synthesis under Multi-dictionary Sparse Representation Framework

Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrieval

Learning to Sketch with Shortcut Cycle Consistency

Exploiting an Intermediate Latent Space between Photo and Sketch for Face Photo-Sketch Recognition

An Efficient Transformer Based on Global and Local Self-attention for Face Photo-Sketch Synthesis

Unsupervised Scene Sketch to Photo Synthesis

Cross-Domain Face Sketch Synthesis.

Cross-Modal Visual Correspondences Learning Without External Semantic Information for Zero-Shot Sketch-Based Image Retrieval

Unsupervised Sketch-to-Photo Synthesis Supplementary Material

Cross-Compatible Embedding and Semantic Consistent Feature Construction for Sketch Re-identification