Abstract:With the prevalence of accessible depth sensors, dynamic human body skeletons have attracted much attention as a robust modality for action recognition. Previous methods model skeletons based on RNN or CNN, which has limited expressive power for irregular skeleton joints. While graph convolutional networks (GCN) have been proposed to address irregular graph-structured data, the fundamental graph construction remains challenging. In this paper, we represent skeletons naturally on graphs, and propose a graph regression based GCN (GR-GCN) for skeleton-based action recognition, aiming to capture the spatio-temporal variation in the data. As the graph representation is crucial to graph convolution, we first propose graph regression to statistically learn the underlying graph from multiple observations. In particular, we provide spatio-temporal modeling of skeletons and pose an optimization problem on the graph structure over consecutive frames, which enforces the sparsity of the underlying graph for efficient representation. The optimized graph not only connects each joint to its neighboring joints in the same frame strongly or weakly, but also links with relevant joints in the previous and subsequent frames. We then feed the optimized graph into the GCN along with the coordinates of the skeleton sequence for feature learning, where we deploy high-order and fast Chebyshev approximation of spectral graph convolution. Further, we provide analysis of the variation characterization by the Chebyshev approximation. Experimental results validate the effectiveness of the proposed graph regression and show that the proposed GR-GCN achieves the state-of-the-art performance on the widely used NTU RGB+D, UT-Kinect and SYSU 3D datasets.

Enhancing human behavior recognition with spatiotemporal graph convolutional neural networks and skeleton sequences

A Multi-Head Pseudo Nodes Based Spatial–temporal Graph Convolutional Network for Emotion Perception from GAIT

Research on Multi-Scale Spatio-Temporal Graph Convolutional Human Behavior Recognition Method Incorporating Multi-Granularity Features

Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

Skeleton-Based Action Recognition with Spatial-Structural Graph Convolution

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Temporal Enhanced Multi-Stream Graph Convolutional Nerual Networks For Skeleton-Based Action Recognition

Human-Robot Collaboration Through a Multi-Scale Graph Convolution Neural Network With Temporal Attention

Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition.

Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

Multiple temporal scale aggregation graph convolutional network for skeleton-based action recognition

Spatio-Temporal Graph Routing for Skeleton-Based Action Recognition

MFGCN: an efficient graph convolutional network based on multi-order feature information for human skeleton action recognition

Spatial and temporal attention embedded spatial temporal graph convolutional networks for skeleton based gait recognition with multiple IMUs

Multi-Scale Adaptive Aggregate Graph Convolutional Network for Skeleton-Based Action Recognition

GraphConvLSTM: Spatiotemporal Learning for Activity Recognition with Wearable Sensors.

Spatiotemporal smoothing aggregation enhanced multi-scale residual deep graph convolutional networks for skeleton-based gait recognition

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

Lightweight Multi-Scale Spatiotemporal Graph Convolutional Network for Skeleton-Based Action Recognition

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Skeleton action recognition via graph convolutional network with self-attention module