A Unified Deep Semantic Expansion Framework for Domain-Generalized Person Re-identification

Eugene P.W. Ang,Shan Lin,Alex C. Kot
DOI: https://doi.org/10.1016/j.neucom.2024.128120
2024-10-11
Abstract:Supervised Person Re-identification (Person ReID) methods have achieved excellent performance when training and testing within one camera network. However, they usually suffer from considerable performance degradation when applied to different camera systems. In recent years, many Domain Adaptation Person ReID methods have been proposed, achieving impressive performance without requiring labeled data from the target domain. However, these approaches still need the unlabeled data of the target domain during the training process, making them impractical in many real-world scenarios. Our work focuses on the more practical Domain Generalized Person Re-identification (DG-ReID) problem. Given one or more source domains, it aims to learn a generalized model that can be applied to unseen target domains. One promising research direction in DG-ReID is the use of implicit deep semantic feature expansion, and our previous method, Domain Embedding Expansion (DEX), is one such example that achieves powerful results in DG-ReID. However, in this work we show that DEX and other similar implicit deep semantic feature expansion methods, due to limitations in their proposed loss function, fail to reach their full potential on large evaluation benchmarks as they have a tendency to saturate too early. Leveraging on this analysis, we propose Unified Deep Semantic Expansion, our novel framework that unifies implicit and explicit semantic feature expansion techniques in a single framework to mitigate this early over-fitting and achieve a new state-of-the-art (SOTA) in all DG-ReID benchmarks. Further, we apply our method on more general image retrieval tasks, also surpassing the current SOTA in all of these benchmarks by wide margins.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve This paper aims to address the performance degradation issue in Domain Generalized Person Re-identification (DG-ReID). Specifically: 1. **Limitations of Existing Methods**: - **Supervised Learning Methods**: Perform well when trained and tested within a single camera network but show significant performance drop when applied to different camera systems. - **Domain Adaptation Methods (DA)**: Achieve good performance without using labeled data from the target domain but still require unlabeled data from the target domain, which is difficult to obtain in many real-world scenarios. 2. **Research Background**: - **Domain Generalization (DG)**: A more practical approach that aims to learn a general model from one or multiple source domains that can be applied to unseen target domains without any target domain data. - **Implicit Deep Semantic Feature Expansion**: A promising research direction that improves the generalization ability of the model by implicitly expanding deep semantic features. The authors' previous method (Domain Embedding Expansion, DEX) achieved strong results in DG-ReID, but its loss function's limitations led to early saturation on large-scale evaluation benchmarks. 3. **Main Contributions of the Paper**: - **Analysis and Improvement of DEX**: A detailed analysis of DEX's limitations revealed that the DEX loss function reduces the inter-class distance in the classifier layer, increasing model complexity and overfitting tendency. - **Proposing the Unified Deep Semantic Expansion Framework (UDSX)**: Combining explicit and implicit semantic feature expansion techniques, the paper introduces three main framework innovations: - **Data Semantic Decoupling (DSD)**: Splits the data flow into two independent paths for explicit and implicit semantic expansion, avoiding mutual interference. - **Progressive Spatio-Temporal Expansion (PSTE)**: Gradually applies explicit semantic expansion in different model layers to stabilize the training process. - **Contrastive-Stream Reunification (CSR)**: Reunifies the two independent semantic expansion streams, ensuring feature invariance across different streams while maintaining intra-class consistency. 4. **Experimental Results**: - **DG-ReID Benchmarks**: UDSX significantly improves test performance on all major DG-ReID benchmarks. - **General Image Retrieval Tasks**: UDSX also significantly outperforms the current state-of-the-art on general image retrieval benchmarks such as CUB-200-2011, Stanford Cars, VehicleID, and Stanford Online Products. ### Summary By analyzing the limitations of existing methods, this paper proposes a new framework, UDSX, which combines explicit and implicit semantic feature expansion techniques to effectively address the performance degradation issue in DG-ReID and achieves significant performance improvements on multiple benchmarks.