Abstract:Hypergraphs have received widespread attention in modeling complex data correlations due to their superior performance. In recent years, some researchers have used hypergraph structures to characterize complex non-pairwise joints in the human skeleton and model higher-order correlations of the human skeleton. However, traditional methods of constructing hypergraphs based on physical connections ignore the dependencies among non-physically connected joints or bones, and it is difficult to model the correlation among joints or bones that are highly correlated in human action but are physically connected at long distances. To address these issues, we propose a skeleton-based action recognition method for hypergraph learning based on skeleton correlation, which explores the effects of physically and non-physically connected skeleton information on accurate action recognition. Specifically, in this paper, spatio-temporal correlation modeling is performed on the natural connections inherent in humans (physical connections) and the joints or bones that are more dependent but not directly connected (non-physical connection) during human actions. In order to better learn the hypergraph structure, we construct a spatio-temporal hypergraph neural network to extract the higher-order correlations of the human skeleton. In addition, we use an attentional mechanism to compute the attentional weights among different hypergraph features, and adaptively fuse the rich feature information in different hypergraphs. Extensive experiments are conducted on two datasets, NTU-RGB+D 60 and Kinetics-Skeleton, and the results show that compared with the state-of-the-art skeleton-based methods, our proposed method can achieve an optimal level of performance with significant advantages, providing a more accurate environmental perception and action analysis for the development of embodied intelligence.

Learning Topological Representation of 3D Skeleton Dynamics with Persistent Homology for Human Activity Recognition

Human motion segmentation using collaborative representations of 3D skeletal sequences.

A Human Activity Recognition Approach Based on Skeleton Extraction and Image Reconstruction

Topological Nonlinear Analysis of Dynamical Systems in Wearable Sensor-Based Human Physical Activity Inference

3d Human Action Recognition Based On The Spatial-Temporal Moving Skeleton Descriptor

Action Recognition Based on Global Optimal Similarity Measuring

Human Action Recognition of Spatiotemporal Parameters for Skeleton Sequences Using MTLN Feature Learning Framework

Toward Fast 3D Human Activity Recognition: A Refined Feature Based on Minimum Joint Freedom Model (Mint)

An effective representation for action recognition with human skeleton joints

Learning Dynamic Spatio-Temporal Relations for Human Activity Recognition.

Skeleton-Based Action Recognition with Spatial-Structural Graph Convolution

Exploring High-Order Skeleton Correlations with Physical and Non-Physical Connection for Action Recognition

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Learning Dictionaries Of Sparse Codes Of 3d Movements Of Body Joints For Real-Time Human Activity Understanding

3D Human Activity Recognition Using Skeletal Data from RGBD Sensors.

Human Action Recognition with Skeleton Induced Discriminative Approximate Rigid Part Model

Action Recognition Based on Motion Representing and Reconstructed Phase Spaces Matching of 3D Joint Positions

Human Activity Recognition Using Dynamic Representation and Matching of Skeleton Feature Sequences from RGB-D Images

Learning to Recognize 3D Human Action from A New Skeleton-based Representation Using Deep Convolutional Neural Networks

Human Activity Recognition with Posture Tendency Descriptors on Action Snippets

Spatial Temporal Network for Image and Skeleton Based Group Activity Recognition