Abstract:With the prevalence of accessible depth sensors, dynamic human body skeletons have attracted much attention as a robust modality for action recognition. Previous methods model skeletons based on RNN or CNN, which has limited expressive power for irregular skeleton joints. While graph convolutional networks (GCN) have been proposed to address irregular graph-structured data, the fundamental graph construction remains challenging. In this paper, we represent skeletons naturally on graphs, and propose a graph regression based GCN (GR-GCN) for skeleton-based action recognition, aiming to capture the spatio-temporal variation in the data. As the graph representation is crucial to graph convolution, we first propose graph regression to statistically learn the underlying graph from multiple observations. In particular, we provide spatio-temporal modeling of skeletons and pose an optimization problem on the graph structure over consecutive frames, which enforces the sparsity of the underlying graph for efficient representation. The optimized graph not only connects each joint to its neighboring joints in the same frame strongly or weakly, but also links with relevant joints in the previous and subsequent frames. We then feed the optimized graph into the GCN along with the coordinates of the skeleton sequence for feature learning, where we deploy high-order and fast Chebyshev approximation of spectral graph convolution. Further, we provide analysis of the variation characterization by the Chebyshev approximation. Experimental results validate the effectiveness of the proposed graph regression and show that the proposed GR-GCN achieves the state-of-the-art performance on the widely used NTU RGB+D, UT-Kinect and SYSU 3D datasets.

RARN: A Real-Time Skeleton-based Action Recognition Network for Auxiliary Rehabilitation Therapy

A Skeleton-based Action Recognition System for Medical Condition Detection

Relational Network for Skeleton-Based Action Recognition

Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognition

Accurate And Real-Time Human Action Recognition Based On 3d Skeleton

View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data

Relation-mining Self-Attention Network for Skeleton-Based Human Action Recognition.

Effective Active Skeleton Representation for Low Latency Human Action Recognition

Action recognition in rehabilitation: combining 3D convolution and LSTM with spatiotemporal attention

Human Skeleton Tree Recurrent Neural Network with Joint Relative Motion Feature for Skeleton Based Action Recognition

Hierarchical recurrent neural network for skeleton based action recognition

Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning

Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition

Recurrent Network Solutions for Human Posture Recognition Based on Kinect Skeletal Data

Asymmetric information-regularized learning for skeleton-based action recognition

Spatio-Temporal Attention Deep Network for Skeleton Based View-Invariant Human Action Recognition

Richly Activated Graph Convolutional Network for Robust Skeleton-Based Action Recognition

Temporal Enhanced Multi-Stream Graph Convolutional Nerual Networks For Skeleton-Based Action Recognition

Spatial Temporal Graph Attention Network for Skeleton-Based Action Recognition

Action Recognition Scheme Based on Skeleton Representation with DS-LSTM Network.

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression