Exploiting Facial Relationships and Feature Aggregation for Multi-Face Forgery Detection
Chenhao Lin,Fangbin Yi,Hang Wang,Jingyi Deng,Zhengyu Zhao,Qian Li,Chao Shen
DOI: https://doi.org/10.1109/tifs.2024.3461469
IF: 7.231
2024-10-02
IEEE Transactions on Information Forensics and Security
Abstract:The emergence of advanced Deepfake technologies has gradually raised concerns in society, prompting significant attention to Deepfake detection. However, in real-world scenarios, Deepfakes often involve multiple faces. Despite this, most existing detection methods still detect these faces individually, overlooking the informative correlation between them and the relationship between the global information of the image and the local information of the faces. In this paper, we address this limitation by proposing FILTER, a novel framework for multi-face forgery detection that explicitly captures underlying correlations. FILTER consists of two main modules: Multi-face Relationship Learning (MRL) and Global Feature Aggregation (GFA). Specifically, MRL learns the correlation of local facial features in multi-face images, and GFA constructs the relationship between image-level labels and individual facial features to enhance performance from a global perspective. In particular, a contrastive learning loss function is used to better discriminate between real and fake faces. Extensive experiments on two publicly available multi-face forgery datasets demonstrate the state-of-the-art performance of FILTER in multi-face forgery detection. For example, on Openforensics Test-Challenge dataset, FILTER outperforms the previous state-of-the-art methods with a higher AUC score (0.980) and higher detection accuracy (92.04%).
computer science, theory & methods,engineering, electrical & electronic