Abstract:Deepfake techniques generate highly realistic data, making it challenging for humans to discern between actual and artificially generated images. Recent advancements in deep learning-based deepfake detection methods, particularly with diffusion models, have shown remarkable progress. However, there is a growing demand for real-world applications to detect unseen individuals, deepfake techniques, and scenarios. To address this limitation, we propose a Prototype-based Unified Framework for Deepfake Detection (PUDD). PUDD offers a detection system based on similarity, comparing input data against known prototypes for video classification and identifying potential deepfakes or previously unseen classes by analyzing drops in similarity. Our extensive experiments reveal three key findings: (1) PUDD achieves an accuracy of 95.1% on Celeb-DF, outperforming state-of-the-art deepfake detection methods; (2) PUDD leverages image classification as the upstream task during training, demonstrating promising performance in both image classification and deepfake detection tasks during inference; (3) PUDD requires only 2.7 seconds for retraining on new data and emits 10$^{5}$ times less carbon compared to the state-of-the-art model, making it significantly more environmentally friendly.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the limitations of current deepfake detection methods in practical applications, which are specifically as follows: 1. **Lack of Robustness**: Most of the existing deepfake detection techniques have difficulty dealing with unseen deepfake samples, that is, those samples created using generation techniques and models different from the training data. This restricts the application of these systems in real - world scenarios. 2. **Time - consuming Training**: Due to the large - scale network models, training these deepfake detection models is very time - consuming. For example, some models need several hours or even days to be retrained to adapt to new individuals or new categories of data. 3. **Lack of Interpretability**: Many detection techniques are difficult to explain their decision - making processes because of their complex network architectures and black - box characteristics. They usually make detection decisions based on high - dimensional feature maps, which limits the transparency and credibility of the model. To solve these problems, the author proposes a Prototype - based Unified Framework for Deepfake Detection (PUDD). By learning prototype information from the original data, PUDD can effectively detect unseen deepfake samples generated using different techniques and models during the inference stage. In addition, PUDD has the following advantages: - **High Efficiency**: PUDD only needs 2.7 seconds to complete retraining on new data, and its carbon emissions are 105 times lower than those of the state - of - the - art models, making it more environmentally friendly. - **Flexibility**: PUDD can be combined with various feature extractors, is suitable for multiple data modalities (such as video and image), and researchers can select appropriate feature extractors according to the specific features of the target category. - **Multi - task Performance**: Although PUDD is mainly designed for deepfake detection and trained based on deepfake datasets, it also performs well in image classification tasks, demonstrating its adaptability and robustness across tasks and datasets. - **Interpretability**: Through prototype learning, PUDD can quantify the reliability of human predictions of model outputs, thus providing better interpretability. In conclusion, PUDD aims to solve the limitations of existing deepfake detection methods in practical applications by improving the robustness, efficiency, flexibility, and interpretability of the detection system.

PUDD: Towards Robust Multi-modal Prototype-based Deepfake Detection

Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes

Multimodal Deepfake Detection for Short Videos

Further research needed if finasteride is to become standard of care for frontal fibrosing alopecia (FFA).

Combating deepfakes: a comprehensive multilayer deepfake video detection framework

Real-Time Advanced Computational Intelligence for Deep Fake Video Detection

An efficient deepfake video detection using robust deep learning

Multi-attentional Deepfake Detection

Unmasking DeepFakes with simple Features

Real-Time Deepfake Video Detection Using Eye Movement Analysis with a Hybrid Deep Learning Approach

DEEPFAKER: A Unified Evaluation Platform for Facial Deepfake and Detection Models

Learning a Deep Dual-Level Network for Robust DeepFake Detection

One Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework

A Multimodal Framework for Deepfake Detection

FFR_FD: Effective and fast detection of DeepFakes via feature point defects

Fully Unsupervised Deepfake Video Detection via Enhanced Contrastive Learning

Deep fake detection using an optimal deep learning model with multi head attention-based feature extraction scheme

A defensive framework for deepfake detection under adversarial settings using temporal and spatial features

Assessment Framework for Deepfake Detection in Real-world Situations

Noise Based Deepfake Detection via Multi-Head Relative-Interaction

Unsupervised Multimodal Deepfake Detection Using Intra- and Cross-Modal Inconsistencies