Regularized forensic efficient net: a game theory based generalized approach for video deepfakes detection

Malik, Khalid Mahmood

DOI: https://doi.org/10.1007/s11042-024-20268-x

IF: 2.577

2024-10-16

Multimedia Tools and Applications

Abstract:Due to the advancements in cutting-edge generative AI algorithms, generating hyper realistic deepfake videos has become easier for the public. This hyperrealism consequently fails contemporary methods to reliably discriminate between original and fake videos. Therefore, to counter any threat caused by these next-generation artificially generated videos, dependable approaches are required to address this classification challenge. To achieve this objective this paper presents an interdisciplinary approach that integrates game theory with deep learning to bring a novel solution to the problem of deepfake detection and protect the detectors against anti-forensics attack. To the best of our knowledge, there does not exist any other work dedicated to video deepfake detection using the integrated approach of game theory and deep learning. The game is designed for two players to distinguish between pristine and deepfake videos. The game utilizes different strategies for the data manipulator as a player P 1 and the deepfake detector as P 2 . Strategies used for P 1 involve the formation of the subsets like open and close-set, combined subsets, imbalanced dataset, and post-processing attacks to create challenging strategies for P 2 . To counter the strategies of P 1 we propose a novel Regularized Forensic Efficient Net (RFE Net) that employs regularization techniques, such as batch normalization, dropout, augmentation, and early stopping. Based on the P 1 move, the detector chooses the regularization techniques by considering factors such as generalizability and efficiency. Regularization-based strategies improve the performance of our model when compared to contemporary methods. Computation of the Nash equilibrium with the proposed zero-sum game helps to effectively detect deepfakes and leads the game to maximum payoff. Performance of the proposed game theory-based RFE Net was measured on standard and diverse datasets of FaceForensic++, DFDC preview, CelebDF, DFFD, and the World leader's dataset, including cross-set and cross-corpus evaluation. Additionally, we also performed the post-processing attacks evaluation and explainability analysis. Experimental results demonstrate that the proposed RFE Net outperforms state-of-the-art methods for deepfakes detection in the defined premises.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering