Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data

Ivan DeAndres-Tame,Ruben Tolosana,Pietro Melzi,Ruben Vera-Rodriguez,Minchul Kim,Christian Rathgeb,Xiaoming Liu,Luis F. Gomez,Aythami Morales,Julian Fierrez,Javier Ortega-Garcia,Zhizhou Zhong,Yuge Huang,Yuxi Mi,Shouhong Ding,Shuigeng Zhou,Shuai He,Lingzhi Fu,Heng Cong,Rongyu Zhang,Zhihong Xiao,Evgeny Smirnov,Anton Pimenov,Aleksei Grigorev,Denis Timoshenko,Kaleb Mesfin Asfaw,Cheng Yaw Low,Hao Liu,Chuyi Wang,Qing Zuo,Zhixiang He,Hatef Otroshi Shahreza,Anjith George,Alexander Unnervik,Parsa Rahimi,Sébastien Marcel,Pedro C. Neto,Marco Huber,Jan Niklas Kolf,Naser Damer,Fadi Boutros,Jaime S. Cardoso,Ana F. Sequeira,Andrea Atzori,Gianni Fenu,Mirko Marras,Vitomir Štruc,Jiang Yu,Zhangjie Li,Jichun Li,Weisong Zhao,Zhen Lei,Xiangyu Zhu,Xiao-Yu Zhang,Bernardo Biesseck,Pedro Vidal,Luiz Coelho,Roger Granada,David Menotti

2024-12-02

Abstract:Synthetic data is gaining increasing popularity for face recognition technologies, mainly due to the privacy concerns and challenges associated with obtaining real data, including diverse scenarios, quality, and demographic groups, among others. It also offers some advantages over real data, such as the large amount of data that can be generated or the ability to customize it to adapt to specific problem-solving needs. To effectively use such data, face recognition models should also be specifically designed to exploit synthetic data to its fullest potential. In order to promote the proposal of novel Generative AI methods and synthetic data, and investigate the application of synthetic data to better train face recognition systems, we introduce the 2nd FRCSyn-onGoing challenge, based on the 2nd Face Recognition Challenge in the Era of Synthetic Data (FRCSyn), originally launched at CVPR 2024. This is an ongoing challenge that provides researchers with an accessible platform to benchmark i) the proposal of novel Generative AI methods and synthetic data, and ii) novel face recognition systems that are specifically proposed to take advantage of synthetic data. We focus on exploring the use of synthetic data both individually and in combination with real data to solve current challenges in face recognition such as demographic bias, domain adaptation, and performance constraints in demanding situations, such as age disparities between training and testing, changes in the pose, or occlusions. Very interesting findings are obtained in this second edition, including a direct comparison with the first one, in which synthetic databases were restricted to DCFace and GANDiffFace.

Computer Vision and Pattern Recognition,Artificial Intelligence,Computers and Society,Machine Learning

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve several key challenges in face recognition technology, especially the problems when using synthetic data to train and improve face recognition systems. Specifically, by organizing the 2nd "Face Recognition Challenge Based on Synthetic Data (2ndFRCSyn - onGoing)", the paper explores the following two main problems:

1. **Demographic Bias Mitigation**:
- The paper attempts to use synthetic data to alleviate the performance differences of face recognition systems among different demographic groups such as races and genders. Traditional real - data sets may lead to lower recognition rates for some groups due to uneven sample distribution, while synthetic data can generate diverse facial images, thus better representing different groups.

2. **Overall Performance Improvement**:
- The paper also explores how to use synthetic data to improve the performance of face recognition systems under various complex conditions, such as age differences, pose changes, and occlusions. These challenges are very common in practical application scenarios, but traditional training methods are often difficult to deal with.

To achieve these goals, the paper introduces a series of subtasks, allowing researchers to use different generation frameworks to create synthetic data and evaluate the impact of these data on face recognition systems. In addition, the paper also provides detailed experimental settings and evaluation metrics to ensure the comparability and reliability of the results.

### Key Research Questions

1. **What are the limitations of face recognition technology trained only with synthetic data?**
- This question explores the performance of systems that rely entirely on synthetic data for training in practical applications and the gap compared to systems trained with real data.

2. **Can synthetic data help alleviate the limitations in current face recognition technology?**
- By comparing the system performance under different conditions (such as using only synthetic data, combining real and synthetic data), the paper evaluates the potential of synthetic data in improving face recognition technology.

The raising and answering of these questions are of great significance for promoting the development of face recognition technology. Especially in the context of increasingly strict privacy protection regulations, synthetic data has become an attractive alternative.