Abstract:Hypergraphs have received widespread attention in modeling complex data correlations due to their superior performance. In recent years, some researchers have used hypergraph structures to characterize complex non-pairwise joints in the human skeleton and model higher-order correlations of the human skeleton. However, traditional methods of constructing hypergraphs based on physical connections ignore the dependencies among non-physically connected joints or bones, and it is difficult to model the correlation among joints or bones that are highly correlated in human action but are physically connected at long distances. To address these issues, we propose a skeleton-based action recognition method for hypergraph learning based on skeleton correlation, which explores the effects of physically and non-physically connected skeleton information on accurate action recognition. Specifically, in this paper, spatio-temporal correlation modeling is performed on the natural connections inherent in humans (physical connections) and the joints or bones that are more dependent but not directly connected (non-physical connection) during human actions. In order to better learn the hypergraph structure, we construct a spatio-temporal hypergraph neural network to extract the higher-order correlations of the human skeleton. In addition, we use an attentional mechanism to compute the attentional weights among different hypergraph features, and adaptively fuse the rich feature information in different hypergraphs. Extensive experiments are conducted on two datasets, NTU-RGB+D 60 and Kinetics-Skeleton, and the results show that compared with the state-of-the-art skeleton-based methods, our proposed method can achieve an optimal level of performance with significant advantages, providing a more accurate environmental perception and action analysis for the development of embodied intelligence.

Human Skeleton Tree Recurrent Neural Network with Joint Relative Motion Feature for Skeleton Based Action Recognition

Hierarchical recurrent neural network for skeleton based action recognition

Graph-Temporal LSTM Networks for Skeleton-Based Action Recognition

Explorations of Skeleton Features for LSTM-based Action Recognition

Relational Network for Skeleton-Based Action Recognition

Skeleton Image Representation for 3D Action Recognition based on Tree Structure and Reference Joints

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates

Fusing Geometric Features for Skeleton-Based Action Recognition Using Multilayer LSTM Networks

View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Learning Local Part Motion Representation for Skeleton-based Action Recognition

Temporal Enhanced Multi-Stream Graph Convolutional Nerual Networks For Skeleton-Based Action Recognition

Multilevel Spatial-Temporal Excited Graph Network for Skeleton-Based Action Recognition

Skeleton-based action recognition with hierarchical spatial reasoning and temporal stack learning network

Deep Stacked Bidirectional Lstm Neural Network For Skeleton-Based Action Recognition

Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition

Exploring High-Order Skeleton Correlations with Physical and Non-Physical Connection for Action Recognition

Joints-Centered Spatial-Temporal Features Fused Skeleton Convolution Network for Action Recognition

MSST-RT: Multi-Stream Spatial-Temporal Relative Transformer for Skeleton-Based Action Recognition

Skeleton-based Multi-Stream Adaptive-Attentional Sub-Graph Convolution Network for Action Recognition

Skeleton-Based Action Recognition With Directed Graph Neural Networks