Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection

Yuzhen Lin,Wentang Song,Bin Li,Yuezun Li,Jiangqun Ni,Han Chen,Qiushi Li

2024-09-22

Abstract:Previous studies in deepfake detection have shown promising results when testing face forgeries from the same dataset as the training. However, the problem remains challenging when one tries to generalize the detector to forgeries from unseen datasets and created by unseen methods. In this work, we present a novel general deepfake detection method, called \textbf{C}urricular \textbf{D}ynamic \textbf{F}orgery \textbf{A}ugmentation (CDFA), which jointly trains a deepfake detector with a forgery augmentation policy network. Unlike the previous works, we propose to progressively apply forgery augmentations following a monotonic curriculum during the training. We further propose a dynamic forgery searching strategy to select one suitable forgery augmentation operation for each image varying between training stages, producing a forgery augmentation policy optimized for better generalization. In addition, we propose a novel forgery augmentation named self-shifted blending image to simply imitate the temporal inconsistency of deepfake generation. Comprehensive experiments show that CDFA can significantly improve both cross-datasets and cross-manipulations performances of various naive deepfake detectors in a plug-and-play way, and make them attain superior performances over the existing methods in several benchmark datasets.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issue in deepfake detection where existing methods perform poorly when dealing with data from unseen datasets or forgeries generated by unknown methods. Specifically, the paper points out that current deepfake detectors achieve good results when the training data and test data distributions are the same. However, in real-world applications, test data often come from unseen datasets and are synthesized by unseen methods, leading to a decline in detector performance.

To tackle this challenge, the paper proposes a new general deepfake detection method called "Curricular Dynamic Forgery Augmentation" (CDFA). CDFA involves jointly training a deepfake detector and a forgery augmentation strategy network, gradually applying forgery augmentation, and dynamically selecting appropriate forgery augmentation operations at different stages of training to optimize the forgery augmentation strategy, thereby improving the detector's generalization ability. Additionally, the paper introduces a new forgery augmentation method—Self-shifted Blending Image (SSBI)—to simulate temporal inconsistencies in deepfake generation.

Through comprehensive experiments, the study shows that CDFA can significantly improve the performance of various baseline deepfake detectors across datasets and operations, surpassing existing methods on multiple benchmark datasets.