Understanding the Security of Deepfake Detection

Xiaoyu Cao,Neil Zhenqiang Gong

DOI: https://doi.org/10.48550/arXiv.2107.02045

2021-10-07

Abstract:Deepfakes pose growing challenges to the trust of information on the Internet. Thus, detecting deepfakes has attracted increasing attentions from both academia and industry. State-of-the-art deepfake detection methods consist of two key components, i.e., face extractor and face classifier, which extract the face region in an image and classify it to be real/fake, respectively. Existing studies mainly focused on improving the detection performance in non-adversarial settings, leaving security of deepfake detection in adversarial settings largely unexplored. In this work, we aim to bridge the gap. In particular, we perform a systematic measurement study to understand the security of the state-of-the-art deepfake detection methods in adversarial settings. We use two large-scale public deepfakes data sources including FaceForensics++ and Facebook Deepfake Detection Challenge, where the deepfakes are fake face images; and we train state-of-the-art deepfake detection methods. These detection methods can achieve 0.94--0.99 accuracies in non-adversarial settings on these datasets. However, our measurement results uncover multiple security limitations of the deepfake detection methods in adversarial settings. First, we find that an attacker can evade a face extractor, i.e., the face extractor fails to extract the correct face regions, via adding small Gaussian noise to its deepfake images. Second, we find that a face classifier trained using deepfakes generated by one method cannot detect deepfakes generated by another method, i.e., an attacker can evade detection via generating deepfakes using a new method. Third, we find that an attacker can leverage backdoor attacks developed by the adversarial machine learning community to evade a face classifier. Our results highlight that deepfake detection should consider the adversarial nature of the problem.

Cryptography and Security,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to understand the security of deep - fake detection methods in an adversarial environment. Specifically, the paper focuses on the performance of the current state - of - the - art deep - fake detection methods when facing attackers who may use various means to evade detection. Through a systematic measurement study, the paper explores the following three main questions:

1. **Q1**: Can attackers deceive the face extractor by adding small perturbations (such as Gaussian noise) to deep - fake images so that it cannot correctly extract the face area?
2. **Q2**: Can a face classifier trained with a certain deep - fake generation method correctly classify deep - fake images generated by other methods?
3. **Q3**: Is the face classifier vulnerable to data - poisoning attacks, especially backdoor attacks?

Through the research on these questions, the paper reveals several security limitations of the current deep - fake detection methods in an adversarial environment and emphasizes the importance of considering the adversarial nature when designing deep - fake detection systems.