Abstract:The fundamental task of zero-shot skeleton-based action recognition is to learn existing skeletal actions during the training phase and to accurately identify unseen actions during the inference phase. The key challenge lies in effectively unifying skeletal features and semantic features. Traditional zero-shot skeleton recognition methods often emphasize the distributed alignment between skeletal and semantic information, thereby neglecting the crucial role that rich semantic and action information plays in guiding skeleton-based action recognition. To enable the model to better focus on semantic features, this paper proposes a Multi-Semantic Embedding (MSE) module. This method aims to more clearly distinguish between different semantic information by using various textual descriptions as semantic guidance, thus improving the efficiency of skeleton feature learning. Additionally, to further enhance the model's generalization ability, this paper introduces a Key Frame Masking (KFM) module. The KFM module selects key frames by calculating the unidirectional motion between each frame in the skeletal sequence. By masking these key frames, it generates sequence samples with information loss, which are then compared with the negative samples. To enable the model to better focus on semantic features and reduce its reliance on the information inherent in the skeletal sequence's key frames. The proposed method was evaluated on three skeletal datasets, achieving an accuracy of 83.34% on the NTU-RGB+D 60 dataset, 57.60% on the NTU-RGB+D 120 dataset, and 78.62% on the PKU-MMD dataset. The experimental results validate the effectiveness of the proposed method in enhancing zero-shot skeleton action recognition.

Improved Key Poses Model for Skeleton-Based Action Recognition.

An optimization method of human skeleton keyframes selection for action recognition

Online Robust Action Recognition Based on a Hierarchical Model

Action Recognition Based on Global Optimal Similarity Measuring

Shifting Perspective to See Difference: A Novel Multi-View Method for Skeleton Based Action Recognition

Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformation

An Approach to Pose-Based Action Recognition

Modelling Human Body Pose for Action Recognition Using Deep Neural Networks

MKTZ: multi-semantic embedding and key frame masking techniques for zero-shot skeleton action recognition

An effective representation for action recognition with human skeleton joints

Fusing Shape and Motion Matrices for View Invariant Action Recognition Using 3D Skeletons

Mining 3d Key-Pose-Motifs for Action Recognition

Skeleton-based action recognition by part-aware graph convolutional networks

Kpose: A New Representation For Action Recognition

Human posture estimation and action recognition on fitness behavior and fitness

Accurate And Real-Time Human Action Recognition Based On 3d Skeleton

Action Recognition from Arbitrary Views Using 3D-Key-pose Set

Deep Progressive Reinforcement Learning For Skeleton-Based Action Recognition

Effective Active Skeleton Representation for Low Latency Human Action Recognition

Skepxels: Spatio-temporal Image Representation of Human Skeleton Joints for Action Recognition

Mining Mid-level Features for Action Recognition Based on Effective Skeleton Representation