SkeletonGait: Gait Recognition Using Skeleton Maps

Chao Fan,Jingzhe Ma,Dongyang Jin,Chuanfu Shen,Shiqi Yu

2023-12-18

Abstract:The choice of the representations is essential for deep gait recognition methods. The binary silhouettes and skeletal coordinates are two dominant representations in recent literature, achieving remarkable advances in many scenarios. However, inherent challenges remain, in which silhouettes are not always guaranteed in unconstrained scenes, and structural cues have not been fully utilized from skeletons. In this paper, we introduce a novel skeletal gait representation named skeleton map, together with SkeletonGait, a skeleton-based method to exploit structural information from human skeleton maps. Specifically, the skeleton map represents the coordinates of human joints as a heatmap with Gaussian approximation, exhibiting a silhouette-like image devoid of exact body structure. Beyond achieving state-of-the-art performances over five popular gait datasets, more importantly, SkeletonGait uncovers novel insights about how important structural features are in describing gait and when they play a role. Furthermore, we propose a multi-branch architecture, named SkeletonGait++, to make use of complementary features from both skeletons and silhouettes. Experiments indicate that SkeletonGait++ outperforms existing state-of-the-art methods by a significant margin in various scenarios. For instance, it achieves an impressive rank-1 accuracy of over 85% on the challenging GREW dataset. All the source code is available at <a class="link-external link-https" href="https://github.com/ShiqiYu/OpenGait" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper mainly addresses several key issues in the field of gait recognition:

1. **Proposed a new skeleton map representation method**: The paper introduces a new skeleton gait representation method called Skeleton Map. This method transforms human joint coordinates into Gaussian approximated heatmaps, forming a new representation similar to contour images but without precise body structure.

2. **Utilizing skeleton information for gait recognition**: Through the SkeletonGait model, the skeleton map is used to extract structural information, achieving state-of-the-art performance on various gait datasets and revealing the importance of structural features in describing gait.

3. **Combining the advantages of skeleton and contour**: The SkeletonGait++ model is proposed, which is a multi-branch architecture that combines the advantages of skeleton maps and contours. This further improves recognition performance in different scenarios, especially when contour information is unreliable (e.g., under poor lighting or severe occlusion).

4. **Exploring the complementarity of skeleton and contour**: Through experimental analysis on multiple datasets, the paper explores the collaborative and complementary characteristics of skeleton maps and traditional contour images in gait recognition, verifying the advantages of skeleton maps in removing background noise, clothing variations, etc.

In summary, the paper aims to improve existing gait recognition techniques through a novel skeleton map representation method and demonstrates its effectiveness across multiple datasets through experiments.