Abstract:The fake videos generated by deep generation technology pose a potential threat to social stability, which makes it critical to detect fake videos. Although the previous detection methods have achieved high accuracy, the generalization to different datasets and in realistic scenes is not effective. We find several novel temporal and spatial clues. In the frequency domain, the inter-frame differences between the real and fake videos are significantly more obvious than the intra-frame differences. In the shallow texture on the CbCr color channels, the forged areas of the fake videos appear more distinct blurring compared to the real videos. And the optical flow of the real video changes gradually, while the optical flow of the fake video changes drastically. This paper proposes a spatio-temporal dual Transformer network for video forgery detection that integrates spatio-temporal clues with the temporal consistency of consecutive frames to improve generalization. Specifically, an EfficientNet is first used to extract spatial artifacts of shallow textures and high-frequency information. We add a new loss function to EfficientNet to extract more robust face features, as well as introduce an attention mechanism to enhance the extracted features. Next, a Swin Transformer is used to capture the subtle temporal artifacts in inter-frame spectrum difference and the optical flow. A feature interaction module is added to fuse local features and global representations. Finally, another Swin Transformer is used to classify the videos according to the extracted spatio-temporal features. We evaluate our method on datasets such as FaceForensics++, Celeb-DF (v2) and DFDC. Extensive experiments show that the proposed framework has high accuracy and generalization, outperforming the current state-of-the-art methods.

Exploring Complementarity of Global and Local Spatiotemporal Information for Fake Face Video Detection

A Cascade Face Spoofing Detector Based on Face Anti-Spoofing R-CNN and Improved Retinex LBP

Face Anti-Spoofing Via Jointly Modeling Local Texture and Constructed Depth

Multi-feature fusion based face forgery detection with local and global characteristics

Refining Localized Attention Features with Multi-Scale Relationships for Enhanced Deepfake Detection in Spatial-Frequency Domain

Hybrid Spatio-Temporal Network for Face Forgery Detection

Combined spatial and frequency dual stream network for face forgery detection

FST-Net - Exploiting Frequency Spatial Temporal Information for Low-Quality Fake Video Detection.

Face Forgery Detection Via Multi-Feature Fusion and Local Enhancement

Attention-Based Spatial-Temporal Multi-Scale Network for Face Anti-Spoofing

Video Forgery Detection Using Spatio-Temporal Dual Transformer.

Latent Spatiotemporal Adaptation for Generalized Face Forgery Video Detection

Temporal Consistency Based Deep Face Forgery Detection Network.

Investigation in Spatial-Temporal Domain for Face Spoof Detection

Detection of Deepfake Videos Using Long-Distance Attention

Towards Spatio-temporal Collaborative Learning: An End-to-End Deepfake Video Detection Framework.

Dual Fusion Paired Environmental Background and Face Region for Face Anti-Spoofing

Dynamic Difference Learning with Spatio-temporal Correlation for Deepfake Video Detection

FakeTransformer: Exposing Face Forgery From Spatial-Temporal Representation Modeled By Facial Pixel Variations

Video Face Recognition Via Combination of Real-Time Local Features and Temporal-Spatial Cues

Dual-Modality Co-Learning for Unveiling Deepfake in Spatio-Temporal Space.