Abstract:The recent proliferation of fake portrait videos poses direct threats on society, law, and privacy [1]. Believing the fake video of a politician, distributing fake pornographic content of celebrities, fabricating impersonated fake videos as evidence in courts are just a few real world consequences of deep fakes. We present a novel approach to detect synthetic content in portrait videos, as a preventive solution for the emerging threat of deep fakes. In other words, we introduce a deep fake detector. We observe that detectors blindly utilizing deep learning are not effective in catching fake content, as generative models produce formidably realistic results. Our key assertion follows that biological signals hidden in portrait videos can be used as an implicit descriptor of authenticity, because they are neither spatially nor temporally preserved in fake content. To prove and exploit this assertion, we first engage several signal transformations for the pairwise separation problem, achieving 99.39% accuracy. Second, we utilize those findings to formulate a generalized classifier for fake content, by analyzing proposed signal transformations and corresponding feature sets. Third, we generate novel signal maps and employ a CNN to improve our traditional classifier for detecting synthetic content. Lastly, we release an "in the wild" dataset of fake portrait videos that we collected as a part of our evaluation process. We evaluate FakeCatcher on several datasets, resulting with 96%, 94.65%, 91.50%, and 91.07% accuracies, on Face Forensics [2], Face Forensics++ [3], CelebDF [4], and on our new Deep Fakes Dataset respectively. In addition, our approach produces a significantly superior detection rate against baselines, and does not depend on the source, generator, or properties of the fake content. We also analyze signals from various facial regions, under image distortions, with varying segment durations, from different generators, against unseen datasets, and under several dimensionality reduction techniques.

In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking

Eyeblink-based Anti-Spoofing in Face Recognition from a Generic Webcamera

Deep fakes Detection Using Human Eye Blinking Pattern: Deep Vision

CP-EB: Talking Face Generation with Controllable Pose and Eye Blinking Embedding

Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence

FakeTransformer: Exposing Face Forgery From Spatial-Temporal Representation Modeled By Facial Pixel Variations

Exposing Deep Fakes Using Inconsistent Head Poses

FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces

An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detection

MesoNet: a Compact Facial Video Forgery Detection Network

Unmasking DeepFakes with simple Features

Face manipulated deepfake generation and recognition approaches: a survey

Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakes

DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms

Finding AI-Generated Faces in the Wild

Real-Time Deepfake Video Detection Using Eye Movement Analysis with a Hybrid Deep Learning Approach

Video Detection Method Based on Temporal and Spatial Foundations for Accurate Verification of Authenticity

DeepFakes: a New Threat to Face Recognition? Assessment and Detection

FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals

Proposed DeepFake Detection Method Using Multiwavelet Transform