Abstract:Existing methods combining skeleton and silhouette representations demonstrate explicit effectiveness for gait recognition. However, current related methods simply combine the video-level representations of model-based skeleton data and gait silhouettes for retrieval. Therefore, diverse skeleton information is not fully exploited in existing related works: Firstly, the position and movement of bones are not clear from individual silhouettes. This indicates that the frame-level interaction between features of skeletons and silhouettes is critical, which is ignored by previous methods. Secondly, diverse part-level skeleton-guided gait features are not fully captured in existing related approaches. To solve the above issues, we present a novel framework with multi-level skeleton-guided refinement, including frame-level, part-level, and video-level skeleton-guided refinement, for comprehensive skeleton-aided gait representation learning. First, two modules are proposed for frame-level skeleton-guided refinement. Specifically, Visual Skeleton Enhanced Backbone (VSEB) is proposed to visually highlight the global and part-level skeleton regions for the feature of each silhouette frame. Moreover, Cross-Visual-Model Frame-level Interaction (CVMFI) is proposed to further transfer the model-based skeleton information to features of the visual modalities. Secondly, part-level visual and model-based skeleton features are utilized to refine the final gait representation. Concretely, in VSEB, Part Skeleton Enhance Network (PSEN) is proposed to visually enhance the position and movement of part-level skeletons. In addition, Semantic Part Pooling (SPP) is proposed for capturing the model-based skeleton features of different semantic parts. Finally, as the video-level skeleton-guided refinement, multimodal video-level features are combined to boost the final recognition performance. Extensive experimental results on prevailing datasets demonstrate that our approach outperforms most existing methods, including the skeleton-aided multi-modal methods. With the multi-level refinement guided by the skeleton modalities, the framework is expected to provide a deeper understanding of skeleton-aided gait recognition.

A Study on Multi-Feature of Gait Recognition Algorithm

Hybrid Silhouette-skeleton Body Representation for Gait Recognition

A novel gait recognition method via fusing shape and kinematics features

Human Gait Recognition Based on Self-Adaptive Hidden Markov Model

Non-local Gait Feature Extraction and Human Identification

Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes

GaitMA: Pose-guided Multi-modal Feature Fusion for Gait Recognition

Gait Recognition Via Multiple Features and Views Information Fusion

Cross-view Gait Recognition Through Ensemble Learning

Human Identification Based on Gait Manifold

Multi-Biometric Feature Extraction from Multiple Pose Estimation Algorithms for Cross-View Gait Recognition

Multi-Modal Gait Recognition with Unidirectional Cross-modal Alignment

GaitAMR: Cross-view gait recognition via aggregated multi-feature representation

Gait Recognition Using The Temporal Information Of Leg Angles

Robust gait recognition using hybrid descriptors based on Skeleton Gait Energy Image

JointsGait:A model-based Gait Recognition Method based on Gait Graph Convolutional Networks and Joints Relationship Pyramid Mapping

Static and Dynamic Features Analysis from Human Skeletons for Gait Recognition

Gait Recognition With Multi-Level Skeleton-Guided Refinement

New Gait Recognition Method Based on Static and Dynamic Features

Progressive Spatio-Temporal Feature Extraction Model for Gait Recognition

Effective Gait Feature Extraction Using Temporal Fusion and Spatial Partial.