Abstract:Visible-infrared person re-identification (RGB-IR ReID) has now attracted increasing attention due to its surveillance applications under low-light environments. However, the large intra-class variations between different domains are still a challenging issue in the field of computer vision. To address the above issue, we propose a novel adversarial Decoupling and Modality-invariant Representation learning (DMiR) method to explore potential spectrum-invariant yet identity-discriminative representations for cross-modality pedestrians. Our model consists of three key components, including Domain-related Representation Disentanglement (DrRD), Modality-invariant Discriminative Representation (MiDR) and Representation Orthogonal Decorrelation (ROD). First, two subnets named Identity-Net and Domain-Net are designed to extract identity-related features and domain-related features, respectively. Given this two-stream structure, the DrRD is introduced to achieve adversarial decoupling against domain-specific features via a min-max disentanglement process. Specifically, the classification objective function on Domain-Net is minimized to extract spectrum-specific information while maximizing it to reduce domain-specific information. Second, in Identity-Net, we introduce MiDR to enhance intra-class compactness and reduce domain variations by exploring positive and negative pair variations, semantic-wise differences, and pair-wise semantic variations. Finally, the correlation between the two decomposed features, i.e., identity-related features and domain-related features, may lead to the introduction of modal information in identity representations, and vice versa. Therefore, we present the ROD constraint to make the two decomposed features unrelated to each other, which can more effectively separate the two-component features and enhance feature representations. Practically, we construct ROD at the feature-level and parameter-level, and finally select feature-level ROD as the decorrelation strategy because of its superior decorrelation performance. The whole scheme leads to disentangling spectrum-dependent information, as well as purifying identity information. Extensive experiments are carried out on two mainstream RGB-IR ReID datasets, and the results demonstrate the effectiveness of our method.

Dual Adversarial Disentanglement and Deep Representation Decorrelation for NIR-VIS Face Recognition

Orthogonal Modality Disentanglement and Representation Alignment Network for NIR-VIS Face Recognition

Modality-transfer Generative Adversarial Network and Dual-Level Unified Latent Representation for Visible Thermal Person Re-Identification

Adversarial Cross-Spectral Face Completion for NIR-VIS Face Recognition

Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition

Towards Nir-Vis Masked Face Recognition

A NIR-to-VIS face recognition via part adaptive and relation attention module

Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition

Balancing Heterogeneous Image Quality for Improved Cross-Spectral Face Recognition

Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-spectral Hallucination and Low-rank Embedding

Near-infrared and visible light face recognition: a comprehensive survey

Modality-Adaptive Mixup and Invariant Decomposition for RGB-Infrared Person Re-Identification

Coupled adversarial learning for semi-supervised heterogeneous face recognition

Parallel-Structure-based Transfer Learning for Deep NIR-to-VIS Face Recognition.

Heterogeneous Face Recognition with Attention-guided Feature Disentangling.

Adversarial Decoupling and Modality-invariant Representation Learning for Visible-Infrared Person Re-identification

Disentanglement for Discriminative Visual Recognition

Cross-Modal and Multi-Attribute Face Recognition: A Benchmark

Pseudo Label Association and Prototype-Based Invariant Learning for Semi-Supervised NIR-VIS Face Recognition

Rethinking the Domain Gap in Near-infrared Face Recognition

A Bidirectional Conversion Network for Cross-Spectral Face Recognition