Abstract:Most of the existing image style transfer algorithms transfer the whole image style as a whole. Style feature is a set of correlation matrix based on style image, namely Gram matrix. Each matrix is a global description of the style image. This kind of methods can perform well in the insensitive semantic scenes (such as the style transfer between landscape photos), but in the sensitive semantic scenes (such as the style transfer between portrait photos), the problem of semantic mismatch will be highlighted, such as transferring the background texture of the style image to the foreground of the target image. Although the existing research takes the manually annotated semantic image as an input of the algorithm, and then guides the style transfer based on the semantic information, and finally achieves good results in the style transfer between portraits. But there are still two problems: first, semantic images need to be manually annotated, which costs human resources. In practical applications, large-scale image style transfer is often needed. Second, the details of the synthesized image are fuzzy, and the definition is not enough. We propose an image style transfer algorithm based on semantic segmentation to resolve semantic mismatching in image style transfer. Our algorithm extracts the semantic information of style image and content image automatically through a semantic segmentation network and uses the semantic information to guide the style transfer. Our algorithm builds a semantic segmentation network based on mask R-CNN, introduces semantic information, and then makes style transfer on the patch level, realizes the style transfer between similar objects (consistent semantic information). Experiments on Celeba and Wikiart show that our method could automatically extract the semantic information of style image and content image. Compared with the state-of-art approaches in this field, our method can effectively avoid semantic mismatch in the process of image st-le transfer. That is, it can maintain semantic consistency in the process of style transfer.

Hair Style Retrieval by Semantic Mapping on Informative Patches

Struct2Hair: A Hair Shape Descriptor for Hairstyle Modeling.

Hair-GAN: Recovering 3D Hair Structure from a Single Image Using Generative Adversarial Networks

A Compositional Exemplar-Based Model For Hair Segmentation

Ihair Recolorer: Deep Image-to-video Hair Color Transfer

Combining Local and Global Information for Hair Shape Modeling

A Novel Coarse-to-fine Hair Segmentation Method.

Image Style Transfer Algorithm Based on Semantic Segmentation

Hairstyle transfer via manipulating decoupled latent codes of StyleGAN2

“Edutainment 2017” a Visual and Semantic Representation of 3D Face Model for Reshaping Face in Images

HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling

StyleIPSB: Identity-Preserving Semantic Basis of StyleGAN for High Fidelity Face Swapping

Face identification using reference-based features with message passing model

Realistic Human Head Modeling with Hairstyle Reconstruction

Towards harmonized regional style transfer and manipulation for facial images

Towards Unified 3D Hair Reconstruction from Single-View Portraits

Hairmony: Fairness-aware hairstyle classification

HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks

Semantic Context-Aware Image Style Transfer

Hair Color Digitization through Imaging and Deep Inverse Graphics

High-quality hair modeling from a single portrait photo