Abstract:Face anti-spoofing (FAS) and face forgery detection play vital roles in securing face biometric systems from presentation attacks (PAs) and vicious digital manipulation (e.g., deepfakes). Despite satisfactory performance upon large-scale data and powerful deep models, recent advances in face spoofing and forgery detection approaches usually focus on 1) unimodal visual appearance or physiological (i.e., remote photoplethysmography (rPPG)) cues; and 2) separated feature representation for FAS or face forgery detection. On one side, unimodal appearance and rPPG features are respectively vulnerable to high-fidelity face 3D mask and video replay attacks, inspiring us to design reliable multi-modal fusion mechanisms for generalized FAS. On the other side, there are rich common features across FAS and face forgery detection tasks (e.g., periodic rPPG rhythms and vanilla appearance for bonafides), providing solid evidence to design a joint FAS and face forgery detection system in a multi-task learning fashion. In this paper, we establish the first joint face spoofing and forgery detection benchmark using both visual appearance and physiological rPPG cues. To enhance the rPPG periodicity discrimination, we design a two-branch physiological network using both facial spatio-temporal rPPG signal map and its continuous wavelet transformed counterpart as inputs. To mitigate the modality bias and improve the fusion efficacy, we conduct a weighted batch and layer normalization for both appearance and rPPG features before multi-modal fusion. We also investigate prevalent deep models, feature fusion strategies and multi-task learning configurations for joint face spoofing and forgery detection. We find that the generalization capacities of both unimodal (appearance or rPPG) and multi-modal (appearance+rPPG) models can be obviously improved via joint training on these two tasks. We hope this new benchmark will facilitate the future research of both FAS and deepfake detection communities. The codes will be released athttps://github.com/ZitongYu/Benchmarking.

JeFaPaTo -- A joint toolbox for blinking analysis and facial features extraction

Modeling Expressions of Peking Opera Facial Make-Ups

Py-Feat: Python Facial Expression Analysis Toolbox

FME '22: 2ndWorkshop on Facial Micro-Expression: Advanced Techniques for Multi-Modal Facial Expression Analysis

FME '22: 2nd Workshop on Facial Micro-Expression: Advanced Techniques for Multi-Modal Facial Expression Analysis

FME '23: 3rd Facial Micro-Expression Workshop

Automatic facial expression recognition on a single 3D face by exploring shape deformation.

Video Tracked Facial Expression Animation

Video-Based Facial Animation with Detailed Appearance Texture

LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis

Analysis of facial expressions recorded from patients during psychiatric interviews

Automatic Facial Expression Recognition Using Features of Salient Facial Patches

Let's Get the FACS Straight -- Reconstructing Obstructed Facial Features

Emotion pattern detection on facial videos using functional statistics

3D/4D Facial Expression Analysis: an Advanced Annotated Face Model Approach

paraFaceTest: an ensemble of regression tree-based facial features extraction for efficient facial paralysis classification

Automatic 2.5-D Facial Landmarking and Emotion Annotation for Social Interaction Assistance

Face Expression Recognition and Analysis: The State of the Art

Benchmarking Joint Face Spoofing and Forgery Detection With Visual and Physiological Cues

Face-GPS: A Comprehensive Technique for Quantifying Facial Muscle Dynamics in Videos

Automatic Fiducial Points Detection for Multi-facial Expressions via Invariant Features and Multi-layer Kernel Sliding Perceptron