Abstract:Previous deepfake detection methods mostly depend on low-level textural features vulnerable to perturbations and fall short of detecting unseen forgery methods. In contrast, high-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. Motivated by this, we propose a detection method that utilizes high-level semantic features of faces to identify inconsistencies in temporal domain. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video classification network, initialized with a meta-functional face encoder for enriched facial representation. In this way, we can take advantage of both the powerful spatio-temporal model and the high-level semantic information of faces. Furthermore, to leverage easily accessible real face data and guide the model in focusing on spatio-temporal features, we design a Dynamic Video Self-Blending (DVSB) method to efficiently generate training samples with diverse spatio-temporal forgery traces using real facial videos. Based on this, we advance our framework with a two-stage training approach: The first stage employs a novel self-supervised contrastive learning, where we encourage the network to focus on forgery traces by impelling videos generated by the same forgery process to have similar representations. On the basis of the representation learned in the first stage, the second stage involves fine-tuning on face forgery detection dataset to build a deepfake detector. Extensive experiments validates that UniForensics outperforms existing face forgery methods in generalization ability and robustness. In particular, our method achieves 95.3\% and 77.2\% cross dataset AUC on the challenging Celeb-DFv2 and DFDC respectively.

Detecting Facial Manipulated Videos Based on Set Convolutional Neural Networks.

Recurrent Convolutional Strategies for Face Manipulation Detection in Videos

DeepFake detection method based on multi-scale interactive dual-stream network

Exploiting Complementary Dynamic Incoherence for DeepFake Video Detection

MDCF-Net: Multi-Scale Dual-Branch Network for Compressed Face Forgery Detection

Face Manipulation Detection Based on Supervised Multi-Feature Fusion Attention Network

A survey on face forgery detection of Deepfake

Detecting Forged Facial Videos using convolutional neural network

UniForensics: Face Forgery Detection via General Facial Representation

An efficient deepfake video detection using robust deep learning

Transformer-based cascade networks with spatial and channel reconstruction convolution for deepfake detection

Dynamic Difference Learning with Spatio-temporal Correlation for Deepfake Video Detection

Detecting Deepfake-Forged Contents with Separable Convolutional Neural Network and Image Segmentation

MesoNet: a Compact Facial Video Forgery Detection Network

Real-Time Deepfake Video Detection Using Eye Movement Analysis with a Hybrid Deep Learning Approach

Multi-feature fusion based face forgery detection with local and global characteristics

Detecting Deepfake Videos Based on Spatiotemporal Attention and Convolutional LSTM

Real-Time Advanced Computational Intelligence for Deep Fake Video Detection

Face Forgery Detection Based on Facial Region Displacement Trajectory Series

SRTNet: a spatial and residual based two-stream neural network for deepfakes detection

One Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework