A Key-Driven Framework for Identity-Preserving Face Anonymization

Miaomiao Wang,Guang Hua,Sheng Li,Guorui Feng

DOI: https://doi.org/10.14722/ndss.2025.23729

2024-09-05

Abstract:Virtual faces are crucial content in the metaverse. Recently, attempts have been made to generate virtual faces for privacy protection. Nevertheless, these virtual faces either permanently remove the identifiable information or map the original identity into a virtual one, which loses the original identity forever. In this study, we first attempt to address the conflict between privacy and identifiability in virtual faces, where a key-driven face anonymization and authentication recognition (KFAAR) framework is proposed. Concretely, the KFAAR framework consists of a head posture-preserving virtual face generation (HPVFG) module and a key-controllable virtual face authentication (KVFA) module. The HPVFG module uses a user key to project the latent vector of the original face into a virtual one. Then it maps the virtual vectors to obtain an extended encoding, based on which the virtual face is generated. By simultaneously adding a head posture and facial expression correction module, the virtual face has the same head posture and facial expression as the original face. During the authentication, we propose a KVFA module to directly recognize the virtual faces using the correct user key, which can obtain the original identity without exposing the original face image. We also propose a multi-task learning objective to train HPVFG and KVFA. Extensive experiments demonstrate the advantages of the proposed HPVFG and KVFA modules, which effectively achieve both facial anonymity and identifiability.

Cryptography and Security,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the conflicting issue of preserving identifiability while protecting privacy in virtual faces. Specifically:

1. **Conflict between Privacy and Identifiability**: Existing virtual face methods either permanently remove identity information or convert the original identity into a virtual identity, leading to the permanent loss of the original identity, which poses an obstacle to face management.
2. **Difficulty in Maintaining Head Pose and Facial Expressions**: To enhance the interaction between the virtual world and the real world, virtual faces need to maintain a certain level of authenticity, including synchronizing head poses and facial expressions to perform tasks such as turning heads, smiling, and blinking.

To solve these problems, the paper proposes a key-based identity protection framework (KFAAR), which includes two modules: the Head Pose Preserving Virtual Face Generation module (HPVFG) and the Key-controllable Virtual Face Authentication module (KVFA). The HPVFG module uses a user-specific key to map the latent vector of the original face into the virtual space, ensuring that the generated virtual face is visually significantly different from the original face while maintaining the original head pose and expressions. The KVFA module is used for the authentication of virtual faces, where, given the correct key, it can extract the original identity of the virtual face for authentication without exposing the visual information of the original face. By training these two modules with a multi-task learning objective, the dual goals of anonymity and identifiability are effectively achieved.