Abstract:A modality separation approach for facial sketch synthesis that addresses the limitations of traditional methods by effectively filtering out irrelevant information and enhancing high‐frequency textures is proposed. This method demonstrates superior performance on the CUHK, AR, and XM2VTS datasets, producing realistic and detailed sketches compared to existing techniques. The technology for face‐to‐sketch synthesis transforms optical face images into a sketch‐style format. However, traditional style losses are insufficient to discern the modal differences between optical and sketch domain images, leading to unclear images. At the same time, generated images lack clarity due to traditional approaches' disregard for high‐frequency texture. To address these issues, a modality separation approach for facial sketch synthesis is proposed. First, a modality separation structure is proposed, using a quicksort algorithm to merge features of optical and sketch images as target modality (positive samples), ensuring the generated images' feature distribution matches real sketches. By controlling the Euclidean distance between generated images (anchors) and both target and filtered modality (positive and negative samples), irrelevant information is effectively filtered out. Next, an edge‐promoting module feeds processed blurry sketch images into the discriminator to enhance robustness. Lastly, a detail optimization module uses Laplacian filtering to extract high‐frequency texture from optical face images for local enhancement. Experimental validation on CUHK, AR, and XM2VTS datasets shows that this method outperforms mainstream sketch face synthesis methods in terms of Fréchet inception distance and learned perceptual image patch similarity, producing more realistic and natural images with richer texture details.

Sketch realizing: Lifelike portrait synthesis from sketch

Multiple Representations-Based Face Sketch–Photo Synthesis

Face sketch-to-photo synthesis from simple line drawing.

DeepPortraitDrawing: Generating human body images from freehand sketches

Deep Generation of Face Images from Sketches

Learning to Sketch Human Facial Portraits Using Personal Styles by Case-Based Reasoning

Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generation

Quality Guided Sketch-to-Photo Image Synthesis

Image-based facial sketch-to-photo synthesis via online coupled dictionary learning

Human-Inspired Facial Sketch Synthesis with Dynamic Adaptation

Sketch-to-Art: Synthesizing Stylized Art Images From Sketches

Face photo-sketch synthesis via full-scale identity supervision

Unsupervised Scene Sketch to Photo Synthesis

Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

Toward Realistic Face Photo–Sketch Synthesis via Composition-Aided GANs

Face Sketch–Photo Synthesis and Retrieval Using Sparse Representation

Learning 3D Face Reconstruction from a Single Sketch

Composition-Aided Face Photo-Sketch Synthesis.

A modality separation approach for facial sketch synthesis

Image based Face Sketch-to-Photo via Online Coupled Dictionary Learning

Automatic Image-Based Pencil Sketch Rendering