Abstract:Existing methods combining skeleton and silhouette representations demonstrate explicit effectiveness for gait recognition. However, current related methods simply combine the video-level representations of model-based skeleton data and gait silhouettes for retrieval. Therefore, diverse skeleton information is not fully exploited in existing related works: Firstly, the position and movement of bones are not clear from individual silhouettes. This indicates that the frame-level interaction between features of skeletons and silhouettes is critical, which is ignored by previous methods. Secondly, diverse part-level skeleton-guided gait features are not fully captured in existing related approaches. To solve the above issues, we present a novel framework with multi-level skeleton-guided refinement, including frame-level, part-level, and video-level skeleton-guided refinement, for comprehensive skeleton-aided gait representation learning. First, two modules are proposed for frame-level skeleton-guided refinement. Specifically, Visual Skeleton Enhanced Backbone (VSEB) is proposed to visually highlight the global and part-level skeleton regions for the feature of each silhouette frame. Moreover, Cross-Visual-Model Frame-level Interaction (CVMFI) is proposed to further transfer the model-based skeleton information to features of the visual modalities. Secondly, part-level visual and model-based skeleton features are utilized to refine the final gait representation. Concretely, in VSEB, Part Skeleton Enhance Network (PSEN) is proposed to visually enhance the position and movement of part-level skeletons. In addition, Semantic Part Pooling (SPP) is proposed for capturing the model-based skeleton features of different semantic parts. Finally, as the video-level skeleton-guided refinement, multimodal video-level features are combined to boost the final recognition performance. Extensive experimental results on prevailing datasets demonstrate that our approach outperforms most existing methods, including the skeleton-aided multi-modal methods. With the multi-level refinement guided by the skeleton modalities, the framework is expected to provide a deeper understanding of skeleton-aided gait recognition.

Robust post-processing strategy for gait silhouette

Hybrid Silhouette-skeleton Body Representation for Gait Recognition

Silhouette Quality Quantification for Gait Sequence Analysis and Recognition.

QAGait: Revisit Gait Recognition from a Quality Perspective

GaitRef: Gait Recognition with Refined Sequential Skeletons

Gait recognition based on multiple image silhouette feature combination

MFCF-Gait: Small Silhouette-Sensitive Gait Recognition Algorithm Based on Multi-Scale Feature Cross-Fusion

Cross-view Gait Recognition Through Ensemble Learning

Silhouette-based Gait Recognition Via Deterministic Learning.

Gait Recognition With Multi-Level Skeleton-Guided Refinement

Combining Procrustes Shape Analysis and Shape Context Descriptor for Silhouette-Based Gait Recognition

A Survey of Silhouette Detection Techniques for Non-Photorealistic Rendering

GaitSTR: Gait Recognition with Sequential Two-stream Refinement

Robust gait recognition using hybrid descriptors based on Skeleton Gait Energy Image

A Novel Algorithm of Gait Recognition

Gait Recognition Using Multichannel Convolution Neural Networks

End-to-End Model-Based Gait Recognition.

CLASH: Complementary Learning with Neural Architecture Search for Gait Recognition

RSANet: Relative-sequence quality assessment network for gait recognition in the wild

GaitContour: Efficient Gait Recognition based on a Contour-Pose Representation

ZipGait: Bridging Skeleton and Silhouette with Diffusion Model for Advancing Gait Recognition