Abstract:With the prevalence of dual-mode cameras in surveillance systems, visible-infrared person re-identification (VI-ReID) has become an emerging topic. Existing studies of VI-ReID roughly fall into three categories: straightforwardly extracting features, improving loss functions, and conducting visible-infrared modality generation. The generation methods avoid the shortcoming of the former two that training models are generally vulnerable to parameter changes. However, these generation methods are usually based on spatial domain and are unavoidable to damage the original information of images. To tackle these limitations, we propose a novel frequency-domain simulated multispectral (FSMS) modality and visible-FSMS-infrared collaborative learning. FSMS modality consists of three-channel images generated by a channel-level reconstruction of visible images, primarily based on the nonsubsampled contourlet transform (NSCT) cooperating with a lightweight network. The generation exploits crucial spectral information and edge information contained in frequency domain. Then, we design a multi-modality network to conduct the tri-modality collaborative learning where FSMS modality is utilized as an intermediate, thereby preserving the original spatial structure of images. Additionally, a dynamic-weight tri-modality heterogeneous retrieval (THR) loss and a modality-shared classification (MSI) loss are devised to mine discriminative modality-invariant features. A cross-modality invariant (CMI) constraint for further exploring triplet-wise relationships and an intra-modality regularizer for relatively stable convergence are introduced. Finally, experimental results show that our algorithm significantly outperforms the latest state-of-the-arts by 5.7% and 4.4% CMC-1 accuracy on two mainstream benchmark datasets, respectively. And the reasons underlying the observed increase in performance are deeply discussed.

Representation Selective Coupling Via Token Sparsification for Multi-Spectral Object Re-Identification

Contribution-Based Multi-Stream Feature Distance Fusion Method with ${k}$ -Distribution Re-Ranking for Person Re-Identification

TOP-ReID: Multi-spectral Object Re-Identification with Token Permutation

Magic Tokens: Select Diverse Tokens for Multi-modal Object Re-Identification

Multi-Scale Cascading Network with Compact Feature Learning for RGB-Infrared Person Re-Identification.

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Dual Attention Matching Network for Context-Aware Feature Sequence based Person Re-Identification

Multi-spectral Vehicle Re-identification with Cross-directional Consistency Network and a High-quality Benchmark

Learning fused features with parallel training for person re-identification

Cross-Spectrum Dual-Subspace Pairing for RGB-infrared Cross-Modality Person Re-Identification

A Spatial-Spectral Relation-Guided Fusion Network for Multisource Optical RS Image Classification

Selective Sparse Sampling for Fine-Grained Image Recognition

Inter-Intra Modality Knowledge Learning and Clustering Noise Alleviation for Unsupervised Visible-Infrared Person Re-Identification

Visible-Infrared Person Re-Identification Based on Frequency-Domain Simulated Multispectral Modality for Dual-Mode Cameras

Joint sparse-collaborative representation to fuse hyperspectral and multispectral images

Heterogeneous feature-aware Transformer-CNN coupling network for person re-identification

CACFTNet: A Hybrid Cov-Attention and Cross-Layer Fusion Transformer Network for Hyperspectral Image Classification

Cross-Spectral Attention for Unsupervised RGB-IR Face Verification and Person Re-identification

Unsupervised Person Re-Identification with Attention-Guided Fine-Grained Features and Symmetric Contrast Learning

Discriminative multi-scale adjacent feature for person re-identification

Cross-directional consistency network with adaptive layer normalization for multi-spectral vehicle re-identification and a high-quality benchmark