Abstract:Video-based person re-identification (ReID) aims to exploit relevant features from spatial and temporal knowledge. Widely used methods include the part- and attention-based approaches for suppressing irrelevant spatial–temporal features. However, it is still challenging to overcome inconsistencies across video frames due to occlusion and imperfect detection. These mismatches make temporal processing ineffective and create an imbalance of crucial spatial information. To address these problems, we propose the Spatiotemporal Multi-Granularity Aggregation (ST-MGA) method, which is specifically designed to accumulate relevant features with spatiotemporally consistent cues. The proposed framework consists of three main stages: extraction, which extracts spatiotemporally consistent partial information; augmentation, which augments the partial information with different granularity levels; and aggregation, which effectively aggregates the augmented spatiotemporal information. We first introduce the consistent part-attention (CPA) module, which extracts spatiotemporally consistent and well-aligned attentive parts. Sub-parts derived from CPA provide temporally consistent semantic information, solving misalignment problems in videos due to occlusion or inaccurate detection, and maximize the efficiency of aggregation through uniform partial information. To enhance the diversity of spatial and temporal cues, we introduce the Multi-Attention Part Augmentation (MA-PA) block, which incorporates fine parts at various granular levels, and the Long-/Short-term Temporal Augmentation (LS-TA) block, designed to capture both long- and short-term temporal relations. Using densely separated part cues, ST-MGA fully exploits and aggregates the spatiotemporal multi-granular patterns by comparing relations between parts and scales. In the experiments, the proposed ST-MGA renders state-of-the-art performance on several video-based ReID benchmarks (i.e., MARS, DukeMTMC-VideoReID, and LS-VID).

Learning Discriminative Aggregation Network for Video-Based Face Recognition and Person Re-identification

Contribution-Based Multi-Stream Feature Distance Fusion Method With <inline-formula> <tex-math notation="LaTeX">${k}$ </tex-math></inline-formula>-Distribution Re-Ranking for Person Re-Identification

Recurrent Embedding Aggregation Network for Video Face Recognition

Contribution-Based Multi-Stream Feature Distance Fusion Method with ${k}$ -Distribution Re-Ranking for Person Re-Identification

Multi-scale spatio-temporal feature adaptive aggregation for video-based Person Re -identification

Joint Uneven Channel Information Network with Blend Metric Loss for Person Re-Identification

Discriminative feature extraction for video person re-identification via multi-task network

Frame Aggregation and Multi-Modal Fusion Framework for Video-Based Person Recognition

Intra-Clip Aggregation for Video Person Re-Identification

Multitask Multigranularity Aggregation With Global-Guided Attention for Video Person Re-Identification

G-FAN: Graph-Based Feature Aggregation Network for Video Face Recognition.

Adaptive Graph Representation Learning for Video Person Re-identification

Joint Structured Sparsity Regularized Multiview Dimension Reduction for Video-Based Facial Expression Recognition.

Adaptive Feature Aggregation for Video Object Detection

Video-based Person re-identification with parallel correction and fusion of pedestrian area features

Hierarchical Attentive Feature Aggregation for Person Re-Identification

Discriminative multi-scale adjacent feature for person re-identification

Flow-guided Feature Enhancement Network for Video-Based Person Re-Identification

Multi-Granularity Aggregation with Spatiotemporal Consistency for Video-Based Person Re-Identification

Dense 3D-Convolutional Neural Network for Person Re-Identification in Videos

Spatial and Temporal Mutual Promotion for Video-Based Person Re-Identification.