Makeup-Guided Facial Privacy Protection via Untrained Neural Network Priors

Fahad Shamshad,Muzammal Naseer,Karthik Nandakumar

2024-08-21

Abstract:Deep learning-based face recognition (FR) systems pose significant privacy risks by tracking users without their consent. While adversarial attacks can protect privacy, they often produce visible artifacts compromising user experience. To mitigate this issue, recent facial privacy protection approaches advocate embedding adversarial noise into the natural looking makeup styles. However, these methods require training on large-scale makeup datasets that are not always readily available. In addition, these approaches also suffer from dataset bias. For instance, training on makeup data that predominantly contains female faces could compromise protection efficacy for male faces. To handle these issues, we propose a test-time optimization approach that solely optimizes an untrained neural network to transfer makeup style from a reference to a source image in an adversarial manner. We introduce two key modules: a correspondence module that aligns regions between reference and source images in latent space, and a decoder with conditional makeup layers. The untrained decoder, optimized via carefully designed structural and makeup consistency losses, generates a protected image that resembles the source but incorporates adversarial makeup to deceive FR models. As our approach does not rely on training with makeup face datasets, it avoids potential male/female dataset biases while providing effective protection. We further extend the proposed approach to videos by leveraging on temporal correlations. Experiments on benchmark datasets demonstrate superior performance in face verification and identification tasks and effectiveness against commercial FR systems. Our code and models will be available at <a class="link-external link-https" href="https://github.com/fahadshamshad/deep-facial-privacy-prior" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the privacy risks posed by facial recognition systems (FR systems). Specifically, while deep learning-based facial recognition technology has improved recognition performance, it has also raised significant privacy concerns, especially in tracking users without their consent. To tackle this issue, the researchers propose a novel method that achieves natural makeup transfer through untrained neural network priors, thereby protecting users' facial privacy.

The method mentioned in the paper has the following advantages compared to existing technologies:
1. **Naturalness**: The generated images look more natural and do not produce obvious artifacts like other methods.
2. **Black-box Attack Effectiveness**: It can work effectively in unknown, i.e., black-box facial recognition systems.
3. **Gender Bias Issue**: It avoids the potential gender bias problem that comes with training on large-scale makeup datasets.
4. **Video Application**: It is not only applicable to static images but can also be extended to videos, utilizing the temporal correlation between frames to improve efficiency.

To achieve these goals, the authors propose a method called "Deep Facial Privacy Prior" (DFPP). This method optimizes the weights of a randomly initialized neural network only during the testing phase to transfer adversarial makeup effects from a reference image to a source image. This approach does not rely on training with large makeup datasets, thereby reducing dataset bias issues and better protecting users' facial privacy while maintaining high visual naturalness. Experimental results show that DFPP performs excellently in facial verification and recognition tasks, outperforming several existing methods.