Abstract:In this work, we propose a dual-stream structured graph convolution network (DS-SGCN) to solve the skeleton-based action recognition problem. The spatio-temporal coordinates and appearance contexts of the skeletal joints are jointly integrated into the graph convolution learning process on both the video and skeleton modalities. To effectively represent the skeletal graph of discrete joints, we create a structured graph convolution module specifically designed to encode partitioned body parts along with their dynamic interactions in the spatio-temporal sequence. In more detail, we build a set of structured intra-part graphs, each of which can be adopted to represent a distinctive body part (e.g., left arm, right leg, head). The inter-part graph is then constructed to model the dynamic interactions across different body parts; here each node corresponds to an intra-part graph built above, while an edge between two nodes is used to express these internal relationships of human movement. We implement the graph convolution learning on both intra- and inter-part graphs in order to obtain the inherent characteristics and dynamic interactions, respectively, of human action. After integrating the intra- and inter-levels of spatial context/coordinate cues, a convolution filtering process is conducted on time slices to capture these temporal dynamics of human motion. Finally, we fuse two streams of graph convolution responses in order to predict the category information of human action in an end-to-end fashion. Comprehensive experiments on five single/multi-modal benchmark datasets (including NTU RGB+D 60, NTU RGB+D 120, MSR-Daily 3D, N-UCLA, and HDM05) demonstrate that the proposed DS-SGCN framework achieves encouraging performance on the skeleton-based action recognition task.

Skeleton-Based Square Grid for Human Action Recognition With 3D Convolutional Neural Network

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

Investigation of Different Skeleton Features for CNN-based 3D Action Recognition

Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognition

Skeleton-based Human Action Recognition via Convolutional Neural Networks (CNN)

Skeleton-based action recognition by part-aware graph convolutional networks

Generalized Graph Convolutional Networks for Skeleton-based Action Recognition

Joints-Centered Spatial-Temporal Features Fused Skeleton Convolution Network for Action Recognition

Channel attention and multi-scale graph neural networks for skeleton-based action recognition

Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition

Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition

DSDC-GCN: Decoupled Static-Dynamic Co-occurrence Graph Convolutional Networks for Skeleton-Based Action Recognition

Skeleton-based Action Recognition Using LSTM and CNN

Dual-Stream Structured Graph Convolution Network for Skeleton-Based Action Recognition

Skeleton-Based Action Recognition with Gated Convolutional Neural Networks

Multi-Scale Adaptive Aggregate Graph Convolutional Network for Skeleton-Based Action Recognition

Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition.

3D Action Recognition Using Data Visualization and Convolutional Neural Networks.

End-to-end Learning of Deep Convolutional Neural Network for 3D Human Action Recognition

An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

Skepxels: Spatio-temporal Image Representation of Human Skeleton Joints for Action Recognition