Abstract:Abstract Recent advances in artificial intelligence technology have significantly improved facial image manipulation, which is known as Deepfake. Facial image manipulation synthesizes or replaces a region of the face in an image with that of another face. The techniques for facial image manipulation are classified into four categories: (1) entire face synthesis, (2) identity swap, (3) attribute manipulation, and (4) expression swap. Out of them, we focus on expression swap because it effectively manipulates only the expression of the face in the images or videos without creating or replacing the entire face, having advantages for the real-time application. In this study, we propose an evaluation framework of the expression swap models targeting the real-time online class environments. For this, we define three kinds of scenarios according to the portion of the face in the entire image considering actual online class situations: (1) attendance check (Scenario 1), (2) presentation (Scenario 2), and (3) examination (Scenario 3). Considering the manipulation on the online class environments, the framework receives a single source image and a target video and generates the video that manipulates a face of the target video to that in the source image. To this end, we select two models that satisfy the conditions required by the framework: (1) first order model and (2) GANimation. We implement these models in the framework and evaluate their performance for the defined scenarios. Through the quantitative and qualitative evaluation, we observe distinguishing properties of the used two models. Specifically, both models show acceptable results in Scenario 1, where the face occupies a large portion of the image. However, their performances are significantly degraded in Scenarios 2 and 3, where the face occupies less portion of the image; the first order model causes relatively less loss of image quality than GANimation in the result of the quantitative evaluation. In contrast, GANimation has the advantages of representing facial expression changes compared to the first order model. Finally, we devise an architecture for applying the expression swap model to the online video conferencing application in real-time. In particular, by applying the expression swap model to widely used online meeting platforms such as Zoom, Google Meet, and Microsoft Teams, we demonstrate its feasibility for real-time online classes.

The Application and Challenges of Face-swapping Technology in the Film Industry

"My Face, My Rules": Enabling Personalized Protection Against Unacceptable Face Editing.

Legal Issues and Solutions for AI Face-swapping Technology in the Context of the rule of the Law

Face/Off: Changing the face of movies with deepfakes

Research on the Application of Artificial Intelligence in the Film Industry

Practical Digital Disguises: Leveraging Face Swaps to Protect Patient Privacy

Digital Face Manipulation Creation and Detection: A Systematic Review

The ethical application of biometric facial recognition technology

Enriching Facial Anti-Spoofing Datasets via an Effective Face Swapping Framework

An experimental study of animating-based facial image manipulation in online class environments

An Introduction to Digital Face Manipulation

Towards mitigating uncann(eye)ness in face swaps via gaze-centric loss terms

3D-Aware Face Swapping

Swapped Face Detection using Deep Learning and Subjective Assessment

Future Trends in Digital Face Manipulation and Detection

Analysis of face recognition technology based on deep learning

Research on the Displacement Impact of Artificial Intelligence on the Film Industry

Face Swapping under Large Pose Variations: A 3D Model Based Approach

Introducing Explicit Gaze Constraints to Face Swapping

Deepfakes in documentary film production: images of deception in the representation of the real

The development history and future prospects of artificial intelligence in facial recognition technology