Abstract:As Artificial Intelligence applications expand, the evaluation of models faces heightened scrutiny. Ensuring public readiness requires evaluation datasets, which differ from training data by being disjoint and ethically sourced in compliance with privacy regulations. The performance and fairness of face recognition systems depend significantly on the quality and representativeness of these evaluation datasets. This data is sometimes scraped from the internet without user's consent, causing ethical concerns that can prohibit its use without proper releases. In rare cases, data is collected in a controlled environment with consent, however, this process is time-consuming, expensive, and logistically difficult to execute. This creates a barrier for those unable to conjure the immense resources required to gather ethically sourced evaluation datasets. To address these challenges, we introduce the Synthetic Identity Generation pipeline, or SIG, that allows for the targeted creation of ethical, balanced datasets for face recognition evaluation. Our proposed and demonstrated pipeline generates high-quality images of synthetic identities with controllable pose, facial features, and demographic attributes, such as race, gender, and age. We also release an open-source evaluation dataset named ControlFace10k, consisting of 10,008 face images of 3,336 unique synthetic identities balanced across race, gender, and age, generated using the proposed SIG pipeline. We analyze ControlFace10k along with a non-synthetic BUPT dataset using state-of-the-art face recognition algorithms to demonstrate its effectiveness as an evaluation tool. This analysis highlights the dataset's characteristics and its utility in assessing algorithmic bias across different demographic groups.

Ethics-aware face recognition aided by synthetic face images

SynFace: Face Recognition with Synthetic Data

Study on the Generation and Comparative Analysis of Ethnically Diverse Faces for Developing a Multiracial Face Recognition Model

Synthetic Data for the Mitigation of Demographic Biases in Face Recognition

Face Recognition Using Synthetic Face Data

Bias and Diversity in Synthetic-based Face Recognition

Synthetic Data for Face Recognition: Current State and Future Prospects

Training Deep Face Recognition Systems with Synthetic Data

If It's Not Enough, Make It So: Reducing Authentic Data Demand in Face Recognition through Synthetic Faces

The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition

SIG: A Synthetic Identity Generation Pipeline for Generating Evaluation Datasets for Face Recognition

SynFER: Towards Boosting Facial Expression Recognition with Synthetic Data

Can Synthetic Faces Undo the Damage of Dataset Bias to Face Recognition and Facial Landmark Detection?

Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluation

Digi2Real: Bridging the Realism Gap in Synthetic Data Face Recognition via Foundation Models

SDFR: Synthetic Data for Face Recognition Competition

From Eyes to Face Synthesis: a New Approach for Human-Centered Smart Surveillance

SASMU: boost the performance of generalized recognition model using synthetic face dataset

Ethical AI in facial expression analysis: racial bias

FRCSyn-onGoing: Benchmarking and comprehensive evaluation of real and synthetic data to improve face recognition systems

VariFace: Fair and Diverse Synthetic Dataset Generation for Face Recognition