Deepfake detection in videos with multiple faces using geometric-fakeness features

Kirill Vyshegorodtsev,Dmitry Kudiyarov,Alexander Balashov,Alexander Kuzmin

2024-10-10

Abstract:Due to the development of facial manipulation techniques in recent years deepfake detection in video stream became an important problem for face biometrics, brand monitoring or online video conferencing solutions. In case of a biometric authentication, if you replace a real datastream with a deepfake, you can bypass a liveness detection system. Using a deepfake in a video conference, you can penetrate into a private meeting. Deepfakes of victims or public figures can also be used by fraudsters for blackmailing, extorsion and financial fraud. Therefore, the task of detecting deepfakes is relevant to ensuring privacy and security. In existing approaches to a deepfake detection their performance deteriorates when multiple faces are present in a video simultaneously or when there are other objects erroneously classified as faces. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video and its per-frame deepfake scores. To analyze temporal inconsistencies in GFFs between the frames we train a complex deep learning model that outputs a final deepfake prediction. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video. Such videos often occur in practice e.g., in an online video conference. In this case, real faces appearing in a frame together with a deepfake face will significantly affect a deepfake detection and our approach allows to counter this problem. Through extensive experiments we demonstrate that our approach outperforms current state-of-the-art methods on popular benchmark datasets such as FaceForensics++, DFDC, Celeb-DF and WildDeepFake. The proposed approach remains accurate when trained to detect multiple different deepfake generation techniques.

Computer Vision and Pattern Recognition,Cryptography and Security

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to detect Deepfakes in video streams, especially when multiple faces appear in the video simultaneously. With the development of facial manipulation techniques, the detection of Deepfakes in videos has become particularly important. This not only affects the security of facial biometric authentication, but also involves areas such as brand monitoring and online video conferencing solutions. If a real data stream is replaced with a Deepfake data stream during the biometric authentication process, the liveness detection system can be bypassed; using Deepfakes in video conferences can infiltrate private meetings. In addition, Deepfakes can also be used by criminals for blackmail, extortion, and financial fraud, etc. Therefore, detecting Deepfakes is crucial for protecting privacy and security.

Existing Deepfake detection methods have reduced performance when multiple faces appear simultaneously in a video or when there are other objects misclassified as faces. To address this problem, this paper proposes a method based on Geometric - Fakeness Features (GFF). This method can characterize the degree of dynamic existence of faces in the video and the Deepfake score for each frame. To analyze the temporal inconsistency of GFF between different frames, the researchers trained a complex deep - learning model to output the final Deepfake prediction results. Through extensive experiments, the researchers demonstrated that their method performs better on popular benchmark datasets such as FaceForensics++, DFDC, Celeb - DF, and WildDeepFake than the existing state - of - the - art methods when dealing with videos containing multiple faces. In addition, the proposed model still maintains accuracy when trained to detect a variety of different Deepfake generation techniques.