Exploring Homogeneous and Heterogeneous Consistent Label Associations for Unsupervised Visible-Infrared Person ReID

Lingfeng He,De Cheng,Nannan Wang,Xinbo Gao

2024-02-04

Abstract:Unsupervised visible-infrared person re-identification (USL-VI-ReID) aims to retrieve pedestrian images of the same identity from different modalities without annotations. While prior work focuses on establishing cross-modality pseudo-label associations to bridge the modality-gap, they ignore maintaining the instance-level homogeneous and heterogeneous consistency in pseudo-label space, resulting in coarse associations. In response, we introduce a Modality-Unified Label Transfer (MULT) module that simultaneously accounts for both homogeneous and heterogeneous fine-grained instance-level structures, yielding high-quality cross-modality label associations. It models both homogeneous and heterogeneous affinities, leveraging them to define the inconsistency for the pseudo-labels and then minimize it, leading to pseudo-labels that maintain alignment across modalities and consistency within intra-modality structures. Additionally, a straightforward plug-and-play Online Cross-memory Label Refinement (OCLR) module is proposed to further mitigate the impact of noisy pseudo-labels while simultaneously aligning different modalities, coupled with a Modality-Invariant Representation Learning (MIRL) framework. Experiments demonstrate that our proposed method outperforms existing USL-VI-ReID methods, highlighting the superiority of our MULT in comparison to other cross-modality association methods. The code will be available.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to conduct visible - infrared person re - identification under unsupervised conditions (USL - VI - ReID). Specifically, the research objective is to retrieve pedestrian images of the same identity from different modalities (visible - light and infrared images) without labeled data. Although existing methods narrow the modality gap by establishing cross - modality pseudo - label associations, they ignore maintaining instance - level homogeneity and heterogeneity consistency in the pseudo - label space, resulting in relatively rough associations. Therefore, this paper proposes a new method, namely the Modality - Unified Label Transfer (MULT) module, which can simultaneously consider the fine - grained instance - level structures of homogeneity and heterogeneity to generate high - quality cross - modality label associations.

In addition, in order to further reduce the influence of noisy pseudo - labels and align different modalities at the same time, this paper also proposes an Online Cross - Memory Label Refinement (OCLR) module and combines it with the Modality - Invariant Representation Learning (MIRL) framework. Experimental results show that the proposed method is superior to existing USL - VI - ReID methods, especially in generating high - quality label associations.