PAFormer: Part Aware Transformer for Person Re-identification

Hyeono Jung,Jangwon Lee,Jiwon Yoo,Dami Ko,Gyeonghwan Kim

2024-08-12

Abstract:Within the domain of person re-identification (ReID), partial ReID methods are considered mainstream, aiming to measure feature distances through comparisons of body parts between samples. However, in practice, previous methods often lack sufficient awareness of anatomical aspect of body parts, resulting in the failure to capture features of the same body parts across different samples. To address this issue, we introduce \textbf{Part Aware Transformer (PAFormer)}, a pose estimation based ReID model which can perform precise part-to-part comparison. In order to inject part awareness to pose tokens, we introduce learnable parameters called `pose token' which estimate the correlation between each body part and partial regions of the image. Notably, at inference phase, PAFormer operates without additional modules related to body part localization, which is commonly used in previous ReID methodologies leveraging pose estimation models. Additionally, leveraging the enhanced awareness of body parts, PAFormer suggests the use of a learning-based visibility predictor to estimate the degree of occlusion for each body part. Also, we introduce a teacher forcing technique using ground truth visibility scores which enables PAFormer to be trained only with visible parts. A set of extensive experiments show that our method outperforms existing approaches on well-known ReID benchmark datasets.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that some existing partial Re - ID (partial ReID) methods lack a sufficient understanding of human anatomical structures when comparing body part features, resulting in an inability to accurately capture the features of the same body parts in different samples. Specifically:

1. **Problems of existing methods**:
- Existing partial ReID methods fail to effectively perform "part - to - part" comparisons.
- These methods often rely on external pose estimation models or additional body part localization modules, even during the inference stage, increasing the computational burden and complexity.
- They do not handle the occlusion problem properly, which affects the robustness and accuracy of the model.

2. **The method proposed in the paper**:
- A ReID model based on pose estimation - Part Aware Transformer (PAFormer) is introduced to achieve accurate part - to - part comparisons.
- PAFormer introduces a learnable parameter "pose token" to estimate the association between each body part and the local area of the image.
- By directly supervising pose heatmap information into the cross - attention mechanism inside ViT, PAFormer can use human body information more efficiently.
- A learning - based visibility predictor is proposed to estimate the occlusion degree of each body part and is trained with pseudo ground truth visibility scores.

3. **Improvement points**:
- **Pose awareness**: Through pose tokens, PAFormer can learn the features of specific body parts during the training process, thereby improving the accuracy of part - to - part comparisons.
- **No additional modules**: During the inference stage, PAFormer does not require additional body part localization modules, simplifying the model structure and improving efficiency.
- **Occlusion handling**: Through the visibility predictor, PAFormer can effectively handle the occlusion problem and improve the performance of the model in actual scenarios.

In summary, this paper aims to address the deficiencies of existing partial ReID methods in part - to - part comparison, dependence on additional modules, and occlusion handling by introducing PAFormer, thereby improving the performance of the re - identification task.