Abstract:With the rapid progress of generative models, the current challenge in face forgery detection is how to effectively detect realistic manipulated faces from different unseen domains. Though previous studies show that pre-trained Vision Transformer (ViT) based models can achieve some promising results after fully fine-tuning on the Deepfake dataset, their generalization performances are still unsatisfactory. One possible reason is that fully fine-tuned ViT-based models may disrupt the pre-trained features [1, 2] and overfit to some data-specific patterns [3]. To alleviate this issue, we present a \textbf{F}orgery-aware \textbf{A}daptive \textbf{Vi}sion \textbf{T}ransformer (FA-ViT) under the adaptive learning paradigm, where the parameters in the pre-trained ViT are kept fixed while the designed adaptive modules are optimized to capture forgery features. Specifically, a global adaptive module is designed to model long-range interactions among input tokens, which takes advantage of self-attention mechanism to mine global forgery clues. To further explore essential local forgery clues, a local adaptive module is proposed to expose local inconsistencies by enhancing the local contextual association. In addition, we introduce a fine-grained adaptive learning module that emphasizes the common compact representation of genuine faces through relationship learning in fine-grained pairs, driving these proposed adaptive modules to be aware of fine-grained forgery-aware information. Extensive experiments demonstrate that our FA-ViT achieves state-of-the-arts results in the cross-dataset evaluation, and enhances the robustness against unseen perturbations. Particularly, FA-ViT achieves 93.83\% and 78.32\% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation. The code and trained model have been released at: <a class="link-external link-https" href="https://github.com/LoveSiameseCat/FAViT" rel="external noopener nofollow">this https URL</a>.

AGIL-SwinT: Attention-guided Inconsistency Learning for Face Forgery Detection

UIA-ViT: Unsupervised Inconsistency-Aware Method Based on Vision Transformer for Face Forgery Detection.

Unified Video and Image Representation for Boosted Video Face Forgery Detection

Face Forensics in the Wild

Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer

Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection

Exploring Bi-Level Inconsistency Via Blended Images for Generalizable Face Forgery Detection

Learning Forgery Region-Aware and ID-Independent Features for Face Manipulation Detection

Learning Natural Consistency Representation for Face Forgery Video Detection

Forgery Face Detection Via Adaptive Learning from Multiple Experts

Towards Generalizable Detection of Face Forgery Via Self-Guided Model-Agnostic Learning

Collaborative Feature Learning for Fine-grained Facial Forgery Detection and Segmentation

Attention-guided Fine-grained Feature Learning For Robust Face Forgery Detection

A3:Ambiguous Aberrations Captured via Astray-Learning for Facial Forgery Semantic Sublimation

Dual Contrastive Learning for General Face Forgery Detection.

A Temporal Consistency Learning Framework for Face Forgery Detection

Learning to mask: Towards generalized face forgery detection

Latent Spatiotemporal Adaptation for Generalized Face Forgery Video Detection

UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localization

Forgery-Domain-Supervised Deepfake Detection with Non-Negative Constraint.

Adapter-Based Incremental Learning for Face Forgery Detection