CodeDetector: Revealing Forgery Traces with Codebook for Generalized Deepfake Detection

Jiaxin Li,Zhihan Yu,Guibo Luo,Yuesheng Zhu

DOI: https://doi.org/10.1145/3652583.3658082

2024-01-01

Abstract:The malicious use of deepfake technologies poses a significant threat to social security, emphasizing the urgent necessity to advance deepfake detection. Existing detection models tend to overfit specific forgery traces within the training set, resulting in a weak generalization performance on unseen data. Considering that the feature distribution of real images is limited compared to the diverse forgery patterns, capturing the real feature space and treating features outside the distribution as potential forgery traces can mitigate overfitting and enhance the detector's generalization ability. Furthermore, since facial manipulation generates artifacts in pixels and disrupts the global consistency of the data, potential forgery traces at both pixel and feature levels can contribute to detection. In this paper, we propose a novel two-stage deepfake detection model named CodeDetector, which utilizes a codebook to capture the feature space of real faces and obtain potential forgery traces by facial reconstruction for detection. In the codebook learning stage, a codebook is used to capture the real feature distribution. In the detector training stage, we obtain pixel-level and feature-level residuals as potential forgery traces through facial reconstruction on real and fake faces to guide the model's attention to forgery clues. Specifically, we propose a Quantized Residual-Guided Attention Module and a Dual Residual Attention Module, calculating residuals and utilizing the attention mechanism to enhance global feature representation. Additionally, an Indices Prediction Module is introduced to ensure the accuracy of residual guidance that enhances the robustness of reconstruction during detection. Extensive experiments have demonstrated that CodeDetector outperforms state-of-the-art in deepfake detection cross-dataset benchmark.