Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection

Zhongjie Ba,Qingyu Liu,Zhenguang Liu,Shuang Wu,Feng Lin,Li Lu,Kui Ren

2024-03-04

Abstract:Deepfake technology has given rise to a spectrum of novel and compelling applications. Unfortunately, the widespread proliferation of high-fidelity fake videos has led to pervasive confusion and deception, shattering our faith that seeing is believing. One aspect that has been overlooked so far is that current deepfake detection approaches may easily fall into the trap of overfitting, focusing only on forgery clues within one or a few local regions. Moreover, existing works heavily rely on neural networks to extract forgery features, lacking theoretical constraints guaranteeing that sufficient forgery clues are extracted and superfluous features are eliminated. These deficiencies culminate in unsatisfactory accuracy and limited generalizability in real-life scenarios.

Computer Vision and Pattern Recognition,Information Theory

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to address two main issues in deepfake detection:

1. **Overfitting Problem**: Current methods tend to learn specific forgery cues from the training dataset, performing poorly on unseen datasets. This means existing methods may focus on only a few forgery regions, leading to limited generalization ability.

2. **Insufficient Feature Extraction**: Existing works overly rely on neural networks to automatically extract forgery features, lacking strict theoretical guarantees to ensure the extraction of sufficient and relevant label information while eliminating irrelevant information. This may result in features that are insufficient to represent genuine forgery cues.

To tackle these issues, the paper proposes a new framework to improve deepfake detection through the following three designs:
1. **Extracting Multiple Non-overlapping Local Features**: By extracting multiple non-overlapping local representations and fusing them into a globally semantically rich feature, it captures a broader range of forgery cues.
2. **Information Loss Based on Information Bottleneck Theory**: Deriving "Local Information Loss" to ensure the orthogonality of local representations and retain comprehensive task-relevant information.
3. **Global Information Loss**: Further theoretical analysis of mutual information leads to "Global Information Loss," used to fuse local representations and remove irrelevant information.

Experimental results show that this method outperforms existing methods on 5 benchmark datasets, achieving the best performance in both in-dataset and cross-dataset settings. Additionally, the method significantly improves performance on unseen datasets, indicating better generalization ability.