Abstract:Skeleton-based action recognition, as a crucial research direction in computer vision, confronts numerous issues and challenges. Most existing research methods rely heavily on extensive labeled data for training, which significantly constraints their training effectiveness and generalization capability when labeled data is scarce. Consequently, how to integrate labeled and unlabeled data to overcome the limitations imposed by label scarcity has emerged as a pivotal research focus in skeleton-based action recognition. Targeting this label scarcity problem, this paper introduces a semi-supervised skeleton-based action recognition approach leveraging multi-dimensional feature-based graph contrastive learning. Firstly, three feature extractors are devised to extract and exploit the available informative cues from limited data thoroughly. The holistic feature extractor comprises five spatio-temporal graph convolutional blocks and a global average pooling layer. The detailed feature extractor is constructed by stacking the same spatio-temporal graph convolutional blocks, while the relational feature extractor primarily integrates stacked attention graph convolutional blocks and a global average pooling layer. Secondly, the sample relationship construction mechanism in graph contrastive learning is enhanced. A clustering process is employed to formulate soft positive/negative sample pairs based on sample similarity, and a sample connectivity matrix further weights the distances between these pairs, thereby enhancing classification accuracy. Furthermore, a novel loss function grounded in the information bottleneck theory is formulated to guide the model towards learning more robust and efficient skeleton action representations. Experimental evaluations demonstrate the superiority of our proposed method (MDKS) on two datasets, NTU60 and NW-UCLA. Specifically, on the NTU60 dataset, MDKS achieves classification accuracy improvements of 4.7% and 1.9% under the X-sub and X-view evaluation protocols, respectively, compared to the benchmark MAC-Learning method. On the NW-UCLA dataset, MDKS outperforms MAC-Learning by 1.4%, 1.2%, 1.9%, and 1.4% in classification accuracy under different labeled data ratios ranging from 5% to 40%. This work offers novel insights and methodologies for advancing skeleton-based action recognition. Future research will delve into label imbalance, label noise, multi-modal information fusion, and cross-scene generalization capabilities.

MS<SUP>2</SUP>L: Multi-Task Self-Supervised Learning for Skeleton Based Action Recognition

EMS2L: Enhanced Multi-Task Self-Supervised Learning for 3D Skeleton Representation Learning

Multi-scale motion contrastive learning for self-supervised skeleton-based action recognition

Multi-Modality Co-Learning for Efficient Skeleton-based Action Recognition

Actionlet-Dependent Contrastive Learning for Unsupervised Skeleton-Based Action Recognition

Semi-supervised learning for skeleton behavior recognition: A multi-dimensional graph comparison approach

Representation modeling learning with multi-domain decoupling for unsupervised skeleton-based action recognition

Multisource Learning for Skeleton-Based Action Recognition Using Deep LSTM and CNN

Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision

Part Aware Contrastive Learning for Self-Supervised Action Recognition

Self-Supervised 3D Skeleton Representation Learning with Active Sampling and Adaptive Relabeling for Action Recognition

Skeleton-Contrastive 3D Action Representation Learning

Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences

Multi-source Learning for Skeleton -Based Action Recognition Using Deep LSTM Networks

Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition

Improving Self-Supervised Action Recognition from Extremely Augmented Skeleton Sequences

Skeleton-Indexed Deep Multi-Modal Feature Learning for High Performance Human Action Recognition

Unsupervised Representation Learning With Long-Term Dynamics for Skeleton Based Action Recognition

Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition

Self-supervised 3D Skeleton Action Representation Learning with Motion Consistency and Continuity

SSRL: Self-Supervised Spatial-Temporal Representation Learning for 3D Action Recognition