UCF: Uncovering Common Features for Generalizable Deepfake Detection

Zhiyuan Yan,Yong Zhang,Yanbo Fan,Baoyuan Wu

2023-10-28

Abstract:Deepfake detection remains a challenging task due to the difficulty of generalizing to new types of forgeries. This problem primarily stems from the overfitting of existing detection methods to forgery-irrelevant features and method-specific patterns. The latter has been rarely studied and not well addressed by previous works. This paper presents a novel approach to address the two types of overfitting issues by uncovering common forgery features. Specifically, we first propose a disentanglement framework that decomposes image information into three distinct components: forgery-irrelevant, method-specific forgery, and common forgery features. To ensure the decoupling of method-specific and common forgery features, a multi-task learning strategy is employed, including a multi-class classification that predicts the category of the forgery method and a binary classification that distinguishes the real from the fake. Additionally, a conditional decoder is designed to utilize forgery features as a condition along with forgery-irrelevant features to generate reconstructed images. Furthermore, a contrastive regularization technique is proposed to encourage the disentanglement of the common and specific forgery features. Ultimately, we only utilize the common forgery features for the purpose of generalizable deepfake detection. Extensive evaluations demonstrate that our framework can perform superior generalization than current state-of-the-art methods.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the generalization problem in deepfake detection. Specifically, existing deepfake detection methods tend to overfit to features unrelated to forgery (such as background, identity, etc.) and patterns specific to a particular forgery technique. These issues lead to a decline in the performance of these detectors when faced with unknown forgery techniques.

To tackle these challenges, the authors propose a novel decoupling framework that decomposes image information into three independent parts: features unrelated to forgery, features specific to a particular forgery technique, and general forgery features. Through a multi-task learning strategy, including a multi-classification task to predict the forgery technique category and a binary classification task to distinguish between real and fake images, the separation between method-specific forgery features and general forgery features is ensured. Additionally, a conditional decoder is designed, which uses forgery features as conditions, combines them with features unrelated to forgery to generate reconstructed images, and introduces contrastive regularization techniques to further promote the decoupling between general and specific forgery features. Ultimately, detection is performed using only general forgery features, thereby enhancing the model's generalization ability.

Experimental results show that the proposed framework performs excellently on multiple benchmark datasets, particularly outperforming current state-of-the-art methods on unseen test datasets.