Abstract:Recent studies show that models trained on synthetic datasets are able to achieve better generalizable person re-identification (GPReID) performance than that trained on public real-world datasets. On the other hand, due to the limitations of real-world person ReID datasets, it would also be important and interesting to use large-scale synthetic datasets as test sets to benchmark person ReID algorithms. Yet this raises a critical question: is synthetic dataset reliable for benchmarking generalizable person re-identification? In the literature there is no evidence showing this. To address this, we design a method called Pairwise Ranking Analysis (PRA) to quantitatively measure the ranking similarity and perform the statistical test of identical distributions. Specifically, we employ Kendall rank correlation coefficients to evaluate pairwise similarity values between algorithm rankings on different datasets. Then, a non-parametric two-sample Kolmogorov-Smirnov (KS) test is performed for the judgement of whether algorithm ranking correlations between synthetic and real-world datasets and those only between real-world datasets lie in identical distributions. We conduct comprehensive experiments, with ten representative algorithms, three popular real-world person ReID datasets, and three recently released large-scale synthetic datasets. Through the designed pairwise ranking analysis and comprehensive evaluations, we conclude that a recent large-scale synthetic dataset ClonedPerson can be reliably used to benchmark GPReID, statistically the same as real-world datasets. Therefore, this study guarantees the usage of synthetic datasets for both source training set and target testing set, with completely no privacy concerns from real-world surveillance data. Besides, the study in this paper might also inspire future designs of synthetic datasets.

Less is More: Learning from Synthetic Data with Fine-grained Attributes for Person Re-Identification

Taking A Closer Look at Synthesis: Fine-grained Attribute Analysis for Person Re-Identification

Attribute Analysis with Synthetic Dataset for Person Re-Identification

From Synthetic to Real: Unveiling the Power of Synthetic Data for Video Person Re-ID

Semi-Supervised Domain Generalizable Person Re-Identification

Is Synthetic Dataset Reliable for Benchmarking Generalizable Person Re-Identification?

Frustratingly Easy Person Re-Identification: Generalizing Person Re-ID in Practice

TAGPerson: A Target-Aware Generation Pipeline for Person Re-identification

Improving person re-identification by attribute and identity learning

TAGPerson: A Target-Aware Generation Pipeline for Person Re-identification

Synthesizing Efficient Data with Diffusion Models for Person Re-Identification Pre-Training

Unsupervised domain adaptation through synthesis for person re-identification

Alice Benchmarks: Connecting Real World Object Re-Identification with the Synthetic.

Learning refined attribute-aligned network with attribute selection for person re-identification

CCUP: A Controllable Synthetic Data Generation Pipeline for Pretraining Cloth-Changing Person Re-Identification Models

Survey for Person Re-Identification Based on Coarse-to-fine Feature Learning

Exploring Fine-Grained Representation and Recomposition for Cloth-Changing Person Re-Identification

Surpassing Real-World Source Training Data: Random 3D Characters for Generalizable Person Re-Identification

Identity-Seeking Self-Supervised Representation Learning for Generalizable Person Re-identification

When Person Re-identification Meets Changing Clothes