PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer

Wentao Jiang,Si Liu,Chen Gao,Jie Cao,Ran He,Jiashi Feng,Shuicheng Yan

DOI: https://doi.org/10.48550/arXiv.1909.06956

2019-11-26

Abstract:In this paper, we address the makeup transfer task, which aims to transfer the makeup from a reference image to a source image. Existing methods have achieved promising progress in constrained scenarios, but transferring between images with large pose and expression differences is still challenging. Besides, they cannot realize customizable transfer that allows a controllable shade of makeup or specifies the part to transfer, which limits their applications. To address these issues, we propose Pose and expression robust Spatial-aware GAN (PSGAN). It first utilizes Makeup Distill Network to disentangle the makeup of the reference image as two spatial-aware makeup matrices. Then, Attentive Makeup Morphing module is introduced to specify how the makeup of a pixel in the source image is morphed from the reference image. With the makeup matrices and the source image, Makeup Apply Network is used to perform makeup transfer. Our PSGAN not only achieves state-of-the-art results even when large pose and expression differences exist but also is able to perform partial and shade-controllable makeup transfer. We also collected a dataset containing facial images with various poses and expressions for evaluations.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is that the existing makeup transfer methods have poor performance when dealing with images with different poses and expressions, and cannot achieve controllable makeup transfer, such as partial makeup transfer or control of makeup intensity. Specifically, the existing methods perform well when dealing with frontal facial images and neutral expressions, but in practical applications, an ideal makeup transfer method needs to generate high - quality results for images with different poses and expressions as well. In addition, the existing methods cannot achieve user - defined makeup transfer, for example, they can only selectively transfer the makeup effects of specific parts (such as eyeshadow or lipstick), or adjust the intensity of makeup. These limitations affect their application scope, especially in makeup transfer in videos. To address these challenges, the author proposes a novel Pose and expression robust Spatial - aware GAN (PSGAN). PSGAN consists of three main parts: Makeup Distill Network (MDNet), Attentive Makeup Morphing (AMM), and Makeup Apply Network (MANet). Through these components, PSGAN can not only achieve makeup transfer in the presence of large pose and expression differences, but also achieve partial makeup transfer and control of makeup intensity. In addition, the author also collected a facial image dataset containing various poses and expressions to evaluate the effect of PSGAN.

PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer

PSGAN: Pose-Robust Spatial-Aware GAN for Customizable Makeup Transfer.

PSGAN: Pose-Robust Spatial-Aware GAN for Customizable Makeup Transfer

PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal

SRPSGAN: Super-resolution with Pose and Expression Robust Spatial-Aware Generative Adversarial Network for Makeup Transfer

Face Makeup Transfer Based on Generative Adversarial Network.

FrseGAN: Free‐style editable facial makeup transfer based on GAN combined with transformer

SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer

Spatially-invariant Style-codes Controlled Makeup Transfer

Beautify As You Like

RAMT-GAN: Realistic and accurate makeup transfer with generative adversarial network

Makeup Transfer Based on Generative Adversarial Network for Large Angle Spatial Misalignment.

MakeupDiffuse: a double image-controlled diffusion model for exquisite makeup transfer

A robust transformer GAN for unpaired data makeup transfer

SLGAN: Style- and Latent-guided Generative Adversarial Network for Desirable Makeup Transfer and Removal

EleGANt: Exquisite and Locally Editable GAN for Makeup Transfer.

TSEV-GAN: Generative Adversarial Networks with Target-aware Style Encoding and Verification for Facial Makeup Transfer.

DA-GAN: Dual-attention Generative Adversarial Networks for Real-World Exquisite Makeup Transfer

Composite makeup transfer model based on generative adversarial networks

BeautyREC: Robust, Efficient, and Component-Specific Makeup Transfer

BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer