A defensive framework for deepfake detection under adversarial settings using temporal and spatial features

S. Asha,P. Vinod,Varun G. Menon
DOI: https://doi.org/10.1007/s10207-023-00695-x
2023-05-04
International Journal of Information Security
Abstract:Advances in artificial intelligence have led to a surge in digital forensics, resulting in numerous image manipulation and processing tools. Hackers and cybercriminals utilize these techniques to create counterfeit images and videos by placing perturbations on facial traits. We propose a novel defensive framework that employs temporal and spatially aware features to efficiently identify deepfakes. This paper utilizes the facial landmarks in the video to train a self-attenuated VGG16 neural model to obtain the spatial attributes. Further, we generate optical flow feature vectors that extract temporal characteristics from the spatial vector. Another necessity of deepfake detection systems is the need for cross-dataset generalization. We built a custom dataset comprising samples from FaceForensics, Celeb-DF, and Youtube videos. Experimental analysis shows that the system achieves a detection accuracy of 98.4%. We evaluate the robustness of our proposed framework under various adversarial settings, employing the Adversarial Robustness Toolbox, Foolbox, and CleverHans tools. The experimental evaluation shows that the proposed method can classify real and fake videos with an accuracy of 74.27% under diverse holistic conditions. An extensive empirical investigation to evaluate the cross-dataset generalization capacity of the proposed framework is also performed.
computer science, information systems, theory & methods, software engineering
What problem does this paper attempt to address?