FSBI: Deepfakes Detection with Frequency Enhanced Self-Blended Images

Ahmed Abul Hasanaath,Hamzah Luqman,Raed Katib,Saeed Anwar

2024-06-25

Abstract:Advances in deepfake research have led to the creation of almost perfect manipulations undetectable by human eyes and some deepfakes detection tools. Recently, several techniques have been proposed to differentiate deepfakes from realistic images and videos. This paper introduces a Frequency Enhanced Self-Blended Images (FSBI) approach for deepfakes detection. This proposed approach utilizes Discrete Wavelet Transforms (DWT) to extract discriminative features from the self-blended images (SBI) to be used for training a convolutional network architecture model. The SBIs blend the image with itself by introducing several forgery artifacts in a copy of the image before blending it. This prevents the classifier from overfitting specific artifacts by learning more generic representations. These blended images are then fed into the frequency features extractor to detect artifacts that can not be detected easily in the time domain. The proposed approach has been evaluated on FF++ and Celeb-DF datasets and the obtained results outperformed the state-of-the-art techniques with the cross-dataset evaluation protocol.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of deepfake detection. With the progress of deepfake technology, the generated fake images and videos have reached a level that is almost impossible to be recognized by the naked eye or some existing deepfake detection tools. This poses a serious threat to the authenticity and integrity of multimedia content, including but not limited to issues such as fake news, malicious content creation, privacy invasion, and identity theft.

To solve these problems, the paper proposes a new method named **Frequency Enhanced Self - Blended Images (FSBI)** for deepfake detection. Specifically, the FSBI method utilizes Discrete Wavelet Transforms (DWT) to extract discriminative features from Self - Blended Images (SBI) and uses these features to train a Convolutional Neural Network (CNN) model. The following are the main contributions of this method:

1. **Proposing an innovative deepfake detection method**: By introducing Frequency Enhanced Self - Blended Images (FSBI), the ability to detect forgery traces in deepfake images is improved.
2. **Discussing the effectiveness of frequency - transformation - based feature extraction in deepfake detection**: Feature extraction by DWT in the frequency domain helps to capture forgery traces that are difficult to find in the time domain.
3. **Evaluating the generalization ability of the proposed method**: The effectiveness and robustness of the method are verified through cross - dataset evaluation.
4. **Achieving the latest optimal performance on two benchmark datasets**: The experimental results on the FF++ and Celeb - DF datasets show that the FSBI method significantly outperforms existing techniques.

### Method Overview

The FSBI method mainly includes three modules:
- **SBI Generator**: By mixing the original image with itself, a self - blended image (SBI) with certain forgery traces is generated. This process prevents the classifier from over - fitting specific forgery traces and instead learns a more general representation.
- **Frequency Features Generator (FFG)**: A series of Discrete Wavelet Transforms (DWT) are applied to the generated SBI image to extract features in the frequency domain. These features can help detect subtle traces in deepfake images.
- **CNN Classifier**: The pre - trained EfficientNet - B5 model is used to train the extracted features to distinguish between real and fake images.

### Experimental Results

The paper conducted experiments on two well - known datasets (FF++ and Celeb - DF). The results show that the FSBI method not only performs well within a single dataset but also significantly outperforms existing methods in cross - dataset evaluation. In particular, in the test on the Celeb - DF dataset, the FSBI method achieved an AUC (Area Under the Curve) value as high as 95.49%, demonstrating its generalization ability under different data distributions and forgery techniques.

In conclusion, through the introduction of the FSBI method, this paper effectively solves the key problems in deepfake detection and provides a new solution for maintaining the authenticity of digital media and public trust.