Abstract:With the rapid development of face forgery techniques, the existing frame-based deepfake video detection methods have fell into a dilemma that frame-based methods may fail when encountering extremely realistic images. To overcome the above problem, many approaches attempted to model the spatio-temporal inconsistency of videos to distinguish real and fake videos. However, current works model spatio-temporal inconsistency by combining intra-frame and inter-frame information, but ignore the disturbance caused by facial motions that would limit further improvement in detection performance. To address this issue, we investigate into long and short range inter-frame motions and propose a novel dynamic difference learning method to distinguish between the inter-frame differences caused by face manipulation and the inter-frame differences caused by facial motions in order to model precise spatio-temporal inconsistency for deepfake video detection. Moreover, we elaborately design a dynamic fine-grained difference capture module (DFDC-module) and a multi-scale spatio-temporal aggregation module (MSA-module) to collaboratively model spatio-temporal inconsistency. Specifically, the DFDC-module applies self-attention mechanism and fine-grained denoising operation to eliminate the differences caused by facial motions and generates long range difference attention maps. The MSA-module is devised to aggregate multi-direction and multi-scale temporal information to model spatio-temporal inconsistency. The existing 2D CNNs can be extended into dynamic spatio-temporal inconsistency capture networks by integrating the proposed two modules. Extensive experimental results demonstrate that our proposed algorithm steadily outperforms state-of-the-art methods by a clear margin in different benchmark datasets.

Video Dynamics Detection Using Deep Neural Networks

Real-Time Epileptic Seizure Detection Based on Deep Learning.

Toward Efficient and Adaptive Design of Video Detection System with Deep Neural Networks

Human Action Recognition From Digital Videos Based on Deep Learning.

Traffic Sign Detection With Event Cameras and DCNN

Deep Learning and Hybrid Approaches for Dynamic Scene Analysis, Object Detection and Motion Tracking

Deepfake Detection Based on Temporal Analysis of Facial Dynamics Using LSTM and ResNeXt Architectures

Video Face Detection Based on Improved SSD Model and Target Tracking Algorithm

Deep Video Anomaly Detection: Opportunities and Challenges

Dynamic Difference Learning with Spatio-temporal Correlation for Deepfake Video Detection

Intelligent Detection Method of English Text in Natural Scenes in Video

DyAnNet: A Scene Dynamicity Guided Self-Trained Video Anomaly Detection Network

A real time crime scene intelligent video surveillance systems in violence detection framework using deep learning techniques

Video Event Restoration Based on Keyframes for Video Anomaly Detection

Deep Neural Networks in Video Human Action Recognition: A Review

Learning Dynamics for Video Facial Expression Recognition

Dynamic Human Behavior Pattern Detection and Classification

Detecting Abrupt Scene Change Using Neural Network

Semantic Analysis System to Recognize Moving Objects by Using a Deep Learning Model

Violence detection in videos using deep recurrent and convolutional neural networks

Real-Time Advanced Computational Intelligence for Deep Fake Video Detection