Abstract:Video-based pedestrian re-identification (Re-ID) is used to re-identify the same person across different camera views. One of the key problems is to learn an effective representation for the pedestrian from video. However, it is difficult to learn an effective representation from one single modality of a feature due to complicated issues with video, such as background, occlusion, and blurred scenes. Therefore, there are some studies on fusing multimodal features for video-based pedestrian Re-ID. However, most of these works fuse features at the global level, which is not effective in reflecting fine-grained and complementary information. Therefore, the improvement in performance is limited. To obtain a more effective representation, we propose to learn fine-grained features from different modalities of the video, and then they are aligned and fused at the fine-grained level to capture rich semantic information. As a result, a multimodal token-learning and alignment model (MTLA) is proposed to re-identify pedestrians across camera videos. An MTLA consists of three modules, i.e., a multimodal feature encoder, token-based cross-modal alignment, and correlation-aware fusion. Firstly, the multimodal feature encoder is used to extract the multimodal features from the visual appearance and gait information views, and then fine-grained tokens are learned and denoised from these features. Then, the token-based cross-modal alignment module is used to align the multimodal features at the token level to capture fine-grained semantic information. Finally, the correlation-aware fusion module is used to fuse the multimodal token features by learning the inter- and intra-modal correlation, in which the features refine each other and a unified representation is obtained for pedestrian Re-ID. To evaluate the performance of fine-grained features alignment and fusion, we conduct extensive experiments on three benchmark datasets. Compared with the state-of-art approaches, all the evaluation metrices of mAP and Rank-K are improved by more than 0.4 percentage points.

Enhancing Person Re-Identification by Integrating Gait Biometric

Person Re-identification Network Based on Multi-Level Feature Fusion

Joining Features by Global Guidance with Bi-Relevance Trihard Loss for Person Re-Identification

Gaussian-based Probability Fusion for Person Re-Identification with Taylor Angular Margin Loss

Human Identification Based on Gait Manifold

GaitHF: enhancing appearance-based gait recognition through height fused images

Cross-view Gait Recognition Through Ensemble Learning

Non-local Gait Feature Extraction and Human Identification

Human Gait Recognition Based on Frame-by-Frame Gait Energy Images and Convolutional Long Short-Term Memory

Pedestrian Re-Identification Based on Gait Analysis

Human Gait Recognition by Integrating Motion Feature and Shape Feature

Swiss-System Based Cascade Ranking for Gait-Based Person Re-Identification

Gait recognition based on 3D human body reconstruction and multi-granular feature fusion

Seq-Masks: Bridging the gap between appearance and gait modeling for video-based person re-identification

Gait Recognition for Human Identification Based on Ica and Fuzzy Svm Through Multiple Views Fusion

TriGait: Aligning and Fusing Skeleton and Silhouette Gait Data via a Tri-Branch Network

Learning Efficient Spatial-Temporal Gait Features with Deep Learning for Human Identification.

Pedestrian Re-Identification Based on Fine-Grained Feature Learning and Fusion

Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes

Video-Based Pedestrian Re-Identification by Adaptive Spatio-Temporal Appearance Model

Person Re-identification by analyzing Dynamic Variations in Gait Sequences