Detection of Synthetic Face Images: Accuracy, Robustness, Generalization

Nela Petrzelkova,Jan Cech

2024-06-25

Abstract:An experimental study on detecting synthetic face images is presented. We collected a dataset, called FF5, of five fake face image generators, including recent diffusion models. We find that a simple model trained on a specific image generator can achieve near-perfect accuracy in separating synthetic and real images. The model handles common image distortions (reduced resolution, compression) by using data augmentation. Moreover, partial manipulations, where synthetic images are blended into real ones by inpainting, are identified and the area of the manipulation is localized by a simple model of YOLO architecture. However, the model turned out to be vulnerable to adversarial attacks and does not generalize to unseen generators. Failure to generalize to detect images produced by a newer generator also occurs for recent state-of-the-art methods, which we tested on Realistic Vision, a fine-tuned version of StabilityAI's Stable Diffusion image generator.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem this paper attempts to address is the accuracy, robustness, and generalization ability of detecting synthetic facial images. Specifically:

1. **Accuracy**: Researchers aim to train a simple model to distinguish between synthetic and real images, achieving near-perfect accuracy on specific image generators.
2. **Robustness**: Researchers enhance the model's robustness by using data augmentation techniques to handle common image distortions (such as resolution reduction and compression).
3. **Generalization Ability**: Researchers find that even when trained on multiple different generators, the model performs poorly in detecting images generated by unseen generators. Additionally, the study explores the model's vulnerability to adversarial attacks and its localization ability in partial image manipulations.

The paper analyzes these aspects through experiments, aiming to provide theoretical basis and technical support for developing more effective synthetic image detection methods.