Abstract:In this article, we address the challenging makeup transfer task, aiming to transfer makeup from a reference image to a source image while preserving facial geometry and background consistency. Existing deep neural network-based methods have shown promising results in aligning facial parts and transferring makeup textures. However, they often neglect the facial geometry of the source image, leading to two adverse effects: (1) alterations in geometrically relevant facial features, causing face flattening and loss of personality, and (2) difficulties in maintaining background consistency, as networks cannot clearly determine the face-background boundary. To jointly tackle these issues, we propose the High Fidelity Makeup via two-dimensional (2D) and 3D Identity Preservation Network (IP23-Net), to the best of our knowledge, a novel framework that leverages facial geometry information to generate more realistic results. Our method comprises a 3D Shape Identity Encoder, which extracts identity and 3D shape features. We incorporate a 3D face reconstruction model to ensure the three-dimensional effect of face makeup, thereby preserving the characters’ depth and natural appearance. To preserve background consistency, our Background Correction Decoder automatically predicts an adaptive mask for the source image, distinguishing the foreground and background. In addition to popular benchmarks, we introduce a new large-scale High Resolution Synthetic Makeup Dataset containing 335,230 diverse high-resolution face images to evaluate our method’s generalization ability. Experiments demonstrate that IP23-Net achieves high-fidelity makeup transfer while effectively preserving background consistency. The code will be made publicly available.

POFMakeup: A style transfer method for Peking Opera makeup

Analysis and Synthesis of Peking Opera Facial Make-Ups

GLStyleNet: Exquisite Style Transfer Combining Global and Local Pyramid Features

A real-time interactive system for facial makeup of Peking Opera

Optimal Transport of Deep Feature for Image Style Transfer

Modeling Expressions of Peking Opera Facial Make-Ups

Expressive Facial Style Transfer for Personalized Memes Mimic

Artistic Style Transfer with Internal-external Learning and Contrastive Learning

Personalized facial makeup transfer based on outline correspondence

Portrait style transfer using deep convolutional neural networks and facial segmentation

Exemplar-Based Portrait Style Transfer

Portrait-Aware Artistic Style Transfer.

Semantic Portrait Color Transfer with Internet Images

Makeup like a superstar: Deep Localized Makeup Transfer Network

BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer

Composite makeup transfer model based on generative adversarial networks

Content-Style Decoupling for Unsupervised Makeup Transfer without Generating Pseudo Ground Truth

High Fidelity Makeup Via 2D and 3D Identity Preservation Net

BeautyREC: Robust, Efficient, and Component-Specific Makeup Transfer

PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer

Facial Makeup Transfer Combining Illumination Transfer