Abstract:As the remarkable development of facial manipulation technologies is accompanied by severe security concerns, face forgery detection has become a recent research hotspot. Most existing detection methods train a binary classifier under global supervision to judge real or fake. However, advanced manipulations only perform small-scale tampering, posing challenges to comprehensively capture subtle and local forgery artifacts, especially in high compression settings and cross-dataset scenarios. To address such limitations, we propose a novel framework named Multi-modal Contrastive Classification by Locally Correlated Representations(MC-LCR), for effective face forgery detection. Instead of specific appearance features, our MC-LCR aims to amplify implicit local discrepancies between authentic and forged faces from both spatial and frequency domains. Specifically, we design the shallow style representation block that measures the pairwise correlation of shallow feature maps, which encodes local style information to extract more discriminative features in the spatial domain. Moreover, we make a key observation that subtle forgery artifacts can be further exposed in the patch-wise phase and amplitude spectrum and exhibit different clues. According to the complementarity of amplitude and phase information, we develop a patch-wise amplitude and phase dual attention module to capture locally correlated inconsistencies with each other in the frequency domain. Besides the above two modules, we further introduce the collaboration of supervised contrastive loss with cross-entropy loss. It helps the network learn more discriminative and generalized representations. Through extensive experiments and comprehensive studies, we achieve state-of-the-art performance and demonstrate the robustness and generalization of our method.

Exploring Frequency Attention Learning and Contrastive Learning for Face Forgery Detection.

Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection

Face forgery detection by progressively enhancing spatial and frequency-aware features

Attention-guided Fine-grained Feature Learning For Robust Face Forgery Detection

Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection

MC-LCR: Multi-modal contrastive classification by locally correlated representations for effective face forgery detection

Refining Localized Attention Features with Multi-Scale Relationships for Enhanced Deepfake Detection in Spatial-Frequency Domain

Face Forgery Detection with Long-Range Noise Features and Multilevel Frequency-Aware Clues

Adaptive Face Forgery Detection in Cross Domain.

Face Forgery Detection Based on Fine-grained Clues and Noise Inconsistency

Exploiting Fine-grained Face Forgery Clues via Progressive Enhancement Learning

Learning to Detect Deepfakes via Adaptive Attention and Constrained Difference.

Face Forgery Detection Via Multi-Feature Fusion and Local Enhancement

Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues

MC-LCR: Multimodal contrastive classification by locally correlated representations for effective face forgery detection

Dual Contrastive Learning for General Face Forgery Detection.

Exploring Frequency Adversarial Attacks for Face Forgery Detection

Detection of Deepfake Videos Using Long-Distance Attention

FPC‐Net: Learning to Detect Face Forgery by Adaptive Feature Fusion of Patch Correlation with CG‐Loss

Where to Focus: Central Attention-Based Face Forgery Detection.

Multiple Feature Mining Based on Local Correlation and Frequency Information for Face Forgery Detection