Abstract:Existing methods combining skeleton and silhouette representations demonstrate explicit effectiveness for gait recognition. However, current related methods simply combine the video-level representations of model-based skeleton data and gait silhouettes for retrieval. Therefore, diverse skeleton information is not fully exploited in existing related works: Firstly, the position and movement of bones are not clear from individual silhouettes. This indicates that the frame-level interaction between features of skeletons and silhouettes is critical, which is ignored by previous methods. Secondly, diverse part-level skeleton-guided gait features are not fully captured in existing related approaches. To solve the above issues, we present a novel framework with multi-level skeleton-guided refinement, including frame-level, part-level, and video-level skeleton-guided refinement, for comprehensive skeleton-aided gait representation learning. First, two modules are proposed for frame-level skeleton-guided refinement. Specifically, Visual Skeleton Enhanced Backbone (VSEB) is proposed to visually highlight the global and part-level skeleton regions for the feature of each silhouette frame. Moreover, Cross-Visual-Model Frame-level Interaction (CVMFI) is proposed to further transfer the model-based skeleton information to features of the visual modalities. Secondly, part-level visual and model-based skeleton features are utilized to refine the final gait representation. Concretely, in VSEB, Part Skeleton Enhance Network (PSEN) is proposed to visually enhance the position and movement of part-level skeletons. In addition, Semantic Part Pooling (SPP) is proposed for capturing the model-based skeleton features of different semantic parts. Finally, as the video-level skeleton-guided refinement, multimodal video-level features are combined to boost the final recognition performance. Extensive experimental results on prevailing datasets demonstrate that our approach outperforms most existing methods, including the skeleton-aided multi-modal methods. With the multi-level refinement guided by the skeleton modalities, the framework is expected to provide a deeper understanding of skeleton-aided gait recognition.

Hybrid Silhouette-skeleton Body Representation for Gait Recognition

Accelerometer-based Gait Recognition Via Voting by Signature Points

SkeletonGait: Gait Recognition Using Skeleton Maps

Robust gait recognition using hybrid descriptors based on Skeleton Gait Energy Image

GaitRef: Gait Recognition with Refined Sequential Skeletons

TriGait: Aligning and Fusing Skeleton and Silhouette Gait Data via a Tri-Branch Network

Gait Identification under Surveillance Environment based on Human Skeleton

Human Gait Recognition Based on Frame-by-Frame Gait Energy Images and Convolutional Long Short-Term Memory

Human Gait Recognition Based on Self-Adaptive Hidden Markov Model

Gait recognition based on 3D human body reconstruction and multi-granular feature fusion

Cross-view Gait Recognition Through Ensemble Learning

Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes

GaitPT: Skeletons Are All You Need For Gait Recognition

GaitSTR: Gait Recognition with Sequential Two-stream Refinement

Static and Dynamic Features Analysis from Human Skeletons for Gait Recognition

Non-local Gait Feature Extraction and Human Identification

GaitSet: Cross-view Gait Recognition through Utilizing Gait as a Deep Set

Human Identification Based on Gait Manifold

Human Gait Recognition with Matrix Representation

A Novel Algorithm of Gait Recognition

Gait Recognition With Multi-Level Skeleton-Guided Refinement