MakeupBag: Disentangling Makeup Extraction and Application

Dokhyam Hoshen

DOI: https://doi.org/10.48550/arXiv.2012.02157

2020-12-04

Abstract:This paper introduces MakeupBag, a novel method for automatic makeup style transfer. Our proposed technique can transfer a new makeup style from a reference face image to another previously unseen facial photograph. We solve makeup disentanglement and facial makeup application as separable objectives, in contrast to other current deep methods that entangle the two tasks. MakeupBag presents a significant advantage for our approach as it allows customization and pixel specific modification of the extracted makeup style, which is not possible using current methods. Extensive experiments, both qualitative and numerical, are conducted demonstrating the high quality and accuracy of the images produced by our method. Furthermore, in contrast to most other current methods, MakeupBag tackles both classical and extreme and costume makeup transfer. In a comparative analysis, MakeupBag is shown to outperform current state-of-the-art approaches.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is Automatic Makeup Style Transfer. Specifically, the paper proposes a new method named MakeupBag, which aims to extract new makeup styles from a reference facial image and apply them to another previously unseen facial photo. This task is decomposed into two independent goals:

1. **Makeup Extraction**: Separate the makeup style from the facial photo and distinguish it from the identity characteristics of the wearer. The challenge in this stage is the lack of labeled training data, so the author adopts unsupervised and weakly - supervised methods to achieve makeup segmentation.

2. **Makeup Application**: Apply a specific makeup style to the target facial photo. In this stage, it is necessary to ensure that the makeup style can be naturally integrated into the target face while keeping the identity characteristics of the target face unchanged.

The main advantage of MakeupBag is that it allows customizing and pixel - level modification of the extracted makeup style, which cannot be achieved by other current methods. In addition, unlike most existing methods, MakeupBag can handle the transfer of classic makeup and extreme or costume makeup. Through a large number of experiments, the paper shows the superior performance of its method in generating high - quality and high - precision images.