Identity-driven Three-Player Generative Adversarial Network for Synthetic-based Face Recognition

Jan Niklas Kolf,Tim Rieber,Jurek Elliesen,Fadi Boutros,Arjan Kuijper,Naser Damer
2023-04-30
Abstract:Many of the commonly used datasets for face recognition development are collected from the internet without proper user consent. Due to the increasing focus on privacy in the social and legal frameworks, the use and distribution of these datasets are being restricted and strongly questioned. These databases, which have a realistically high variability of data per identity, have enabled the success of face recognition models. To build on this success and to align with privacy concerns, synthetic databases, consisting purely of synthetic persons, are increasingly being created and used in the development of face recognition solutions. In this work, we present a three-player generative adversarial network (GAN) framework, namely IDnet, that enables the integration of identity information into the generation process. The third player in our IDnet aims at forcing the generator to learn to generate identity-separable face images. We empirically proved that our IDnet synthetic images are of higher identity discrimination in comparison to the conventional two-player GAN, while maintaining a realistic intra-identity variation. We further studied the identity link between the authentic identities used to train the generator and the generated synthetic identities, showing very low similarities between these identities. We demonstrated the applicability of our IDnet data in training face recognition models by evaluating these models on a wide set of face recognition benchmarks. In comparison to the state-of-the-art works in synthetic-based face recognition, our solution achieved comparable results to a recent rendering-based approach and outperformed all existing GAN-based approaches. The training code and the synthetic face image dataset are publicly available ( <a class="link-external link-https" href="https://github.com/fdbtrs/Synthetic-Face-Recognition" rel="external noopener nofollow">this https URL</a> ).
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### The Problem the Paper Attempts to Solve This paper aims to address the privacy issues associated with the datasets used in the development of Face Recognition (FR). Many commonly used datasets are collected from the internet without proper user consent. With the increasing focus on privacy within social and legal frameworks, the use and distribution of these datasets have been restricted and strongly questioned. These real datasets, with high data variability, have made the success of face recognition models possible. To address privacy issues while maintaining this success, there is a growing trend to create and use synthetic databases composed entirely of synthetic individuals. Specifically, this paper proposes a Three-Player Generative Adversarial Network (GAN) framework, called IDnet, which integrates identity information into the generation process. The goal of the third player (ID-3) is to force the generator to learn to generate facial images that are distinguishable in terms of identity. Experiments demonstrate that, compared to traditional two-player GANs, the synthetic images generated by IDnet have higher identity distinguishability while maintaining realistic intra-identity variations. Additionally, the study explores the identity link between real identities used to train the generator and the generated synthetic identities, showing that the similarity between these identities is very low. Finally, by evaluating these models on multiple face recognition benchmarks, the applicability of IDnet data in training face recognition models is demonstrated. Compared to existing face recognition methods based on synthetic data, the proposed solution achieves performance comparable to the latest rendering methods and surpasses all existing GAN methods.