Analysis of Human Perception in Distinguishing Real and AI-Generated Faces: An Eye-Tracking Based Study

Jin Huang,Subhadra Gopalakrishnan,Trisha Mittal,Jake Zuena,Jaclyn Pytlarz

2024-09-24

Abstract:Recent advancements in Artificial Intelligence have led to remarkable improvements in generating realistic human faces. While these advancements demonstrate significant progress in generative models, they also raise concerns about the potential misuse of these generated images. In this study, we investigate how humans perceive and distinguish between real and fake images. We designed a perceptual experiment using eye-tracking technology to analyze how individuals differentiate real faces from those generated by AI. Our analysis of StyleGAN-3 generated images reveals that participants can distinguish real from fake faces with an average accuracy of 76.80%. Additionally, we found that participants scrutinize images more closely when they suspect an image to be fake. We believe this study offers valuable insights into human perception of AI-generated media.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issue of how humans perceive and distinguish between real and AI-generated facial images. Specifically:

1. **Research Background**: With the development of technologies such as Generative Adversarial Networks (GANs), AI-generated facial images are becoming increasingly realistic. This brings many application benefits, such as virtual reality and digital art, but also raises ethical and privacy concerns, such as identity theft and the spread of false information.

2. **Research Objective**: To analyze human behavior patterns in distinguishing real from fake faces through eye-tracking experiments, thereby better understanding how humans recognize these images and providing a basis for further development of relevant ethical guidelines and regulatory frameworks.

3. **Experimental Design**: Researchers used real facial images (from the FFHQ dataset) and fake facial images generated by StyleGAN-3. They recorded participants' eye movements while viewing these images using eye-tracking devices and asked participants to judge the authenticity of the images. The experiment collected data from over 7,000 images.

4. **Main Findings**:
- Participants were able to distinguish real from fake facial images with an average accuracy of 76.80%.
- When participants suspected an image to be fake, they examined the image more carefully.
- Eye-tracking data showed that participants focused more on the head, facial features, and background components when judging the authenticity of the images.

5. **Further Analysis**: By analyzing the impact of different attributes (such as background, gender, skin color, etc.), it was found that the background had the greatest influence on participants' judgment of image authenticity, especially when there were people in the background. Additionally, participants spent more time on real images, indicating that some real images might be harder to distinguish than fake ones.

Overall, this study aims to reveal the cognitive process of humans in recognizing AI-generated facial images through eye-tracking technology, with the goal of providing guidance for the future development of image synthesis technology and helping to establish more reasonable usage norms.