Abstract:Video-based pedestrian re-identification (Re-ID) is used to re-identify the same person across different camera views. One of the key problems is to learn an effective representation for the pedestrian from video. However, it is difficult to learn an effective representation from one single modality of a feature due to complicated issues with video, such as background, occlusion, and blurred scenes. Therefore, there are some studies on fusing multimodal features for video-based pedestrian Re-ID. However, most of these works fuse features at the global level, which is not effective in reflecting fine-grained and complementary information. Therefore, the improvement in performance is limited. To obtain a more effective representation, we propose to learn fine-grained features from different modalities of the video, and then they are aligned and fused at the fine-grained level to capture rich semantic information. As a result, a multimodal token-learning and alignment model (MTLA) is proposed to re-identify pedestrians across camera videos. An MTLA consists of three modules, i.e., a multimodal feature encoder, token-based cross-modal alignment, and correlation-aware fusion. Firstly, the multimodal feature encoder is used to extract the multimodal features from the visual appearance and gait information views, and then fine-grained tokens are learned and denoised from these features. Then, the token-based cross-modal alignment module is used to align the multimodal features at the token level to capture fine-grained semantic information. Finally, the correlation-aware fusion module is used to fuse the multimodal token features by learning the inter- and intra-modal correlation, in which the features refine each other and a unified representation is obtained for pedestrian Re-ID. To evaluate the performance of fine-grained features alignment and fusion, we conduct extensive experiments on three benchmark datasets. Compared with the state-of-art approaches, all the evaluation metrices of mAP and Rank-K are improved by more than 0.4 percentage points.

Cross-Compatible Embedding and Semantic Consistent Feature Construction for Sketch Re-identification

SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches

Contribution-Based Multi-Stream Feature Distance Fusion Method with ${k}$ -Distribution Re-Ranking for Person Re-Identification

Cross-Domain Adversarial Feature Learning For Sketch Re-Identification

Co-attention Aligned Mutual Cross-Attention for Cloth-Changing Person Re-identification.

SketchTrans: Disentangled Prototype Learning With Transformer for Sketch-Photo Recognition

Pedestrian Re-ID based on feature consistency and contrast enhancement

Disentangled body features for clothing change person re-identification

Salience-Guided Cascaded Suppression Network for Person Re-identification

Progressive Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image Retrieval

See What You Seek: Semantic Contextual Integration for Cloth-Changing Person Re-Identification

Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style

UnifiedSC: a unified framework via collaborative optimization for multi-task person re-identification

Cross-Modality Person Re-Identification With Shared-Specific Feature Transfer

Pedestrian Re-Identification Based on Fine-Grained Feature Learning and Fusion

SR-DSFF and FENet-ReID: A Two-Stage Approach for Cross Resolution Person Re-Identification

Heterogeneous feature-aware Transformer-CNN coupling network for person re-identification

X-ReID: Cross-Instance Transformer for Identity-Level Person Re-Identification

A Semantic Perception and CNN-Transformer Hybrid Network for Occluded Person Re-identification

Cross Task Modality Alignment Network for Sketch Face Recognition

Dual-stream feature fusion network for person re-identification