Abstract:Face anti-spoofing (FAS) and face forgery detection play vital roles in securing face biometric systems from presentation attacks (PAs) and vicious digital manipulation (e.g., deepfakes). Despite satisfactory performance upon large-scale data and powerful deep models, recent advances in face spoofing and forgery detection approaches usually focus on 1) unimodal visual appearance or physiological (i.e., remote photoplethysmography (rPPG)) cues; and 2) separated feature representation for FAS or face forgery detection. On one side, unimodal appearance and rPPG features are respectively vulnerable to high-fidelity face 3D mask and video replay attacks, inspiring us to design reliable multi-modal fusion mechanisms for generalized FAS. On the other side, there are rich common features across FAS and face forgery detection tasks (e.g., periodic rPPG rhythms and vanilla appearance for bonafides), providing solid evidence to design a joint FAS and face forgery detection system in a multi-task learning fashion. In this paper, we establish the first joint face spoofing and forgery detection benchmark using both visual appearance and physiological rPPG cues. To enhance the rPPG periodicity discrimination, we design a two-branch physiological network using both facial spatio-temporal rPPG signal map and its continuous wavelet transformed counterpart as inputs. To mitigate the modality bias and improve the fusion efficacy, we conduct a weighted batch and layer normalization for both appearance and rPPG features before multi-modal fusion. We also investigate prevalent deep models, feature fusion strategies and multi-task learning configurations for joint face spoofing and forgery detection. We find that the generalization capacities of both unimodal (appearance or rPPG) and multi-modal (appearance+rPPG) models can be obviously improved via joint training on these two tasks. We hope this new benchmark will facilitate the future research of both FAS and deepfake detection communities. The codes will be released athttps://github.com/ZitongYu/Benchmarking.

Face Anti-spoofing Based on Cooperative Pose Analysis.

A Cascade Face Spoofing Detector Based on Face Anti-Spoofing R-CNN and Improved Retinex LBP

Self-Attention and MLP Auxiliary Convolution for Face Anti-Spoofing

Face Anti-Spoofing: Model Matters, So Does Data

Face Liveness Detection by rPPG Features and Contextual Patch-Based CNN

Face Anti-Spoofing with Human Material Perception

Deep Learning for Face Anti-Spoofing: A Survey

Context based face anti-spoofing

Face Anti-Spoofing by Learning Polarization Cues in a Real-World Scenario

Robust Face Recognition by Constrained Part-based Alignment.

An Effective Face Anti-Spoofing Method via Stereo Matching

Disentangled Representation based Face Anti-Spoofing

Face Anti-Spoofing Using Spatial Pyramid Pooling

Face anti-spoofing with cross-stage relation enhancement and spoof material perception

Benchmarking Joint Face Spoofing and Forgery Detection With Visual and Physiological Cues

A Simple and Effective Patch-Based Method for Frame-Level Face Anti-Spoofing.

CNN-LPQ: convolutional neural network combined to local phase quantization based approach for face anti-spoofing

3D Face Anti-Spoofing With Factorized Bilinear Coding

Facial Anti-Spoofing Using "Clue Maps"

Algorithm of face anti-spoofing based on pseudo-negative features generation