Abstract:In this work, we propose a dual-stream structured graph convolution network (DS-SGCN) to solve the skeleton-based action recognition problem. The spatio-temporal coordinates and appearance contexts of the skeletal joints are jointly integrated into the graph convolution learning process on both the video and skeleton modalities. To effectively represent the skeletal graph of discrete joints, we create a structured graph convolution module specifically designed to encode partitioned body parts along with their dynamic interactions in the spatio-temporal sequence. In more detail, we build a set of structured intra-part graphs, each of which can be adopted to represent a distinctive body part (e.g., left arm, right leg, head). The inter-part graph is then constructed to model the dynamic interactions across different body parts; here each node corresponds to an intra-part graph built above, while an edge between two nodes is used to express these internal relationships of human movement. We implement the graph convolution learning on both intra- and inter-part graphs in order to obtain the inherent characteristics and dynamic interactions, respectively, of human action. After integrating the intra- and inter-levels of spatial context/coordinate cues, a convolution filtering process is conducted on time slices to capture these temporal dynamics of human motion. Finally, we fuse two streams of graph convolution responses in order to predict the category information of human action in an end-to-end fashion. Comprehensive experiments on five single/multi-modal benchmark datasets (including NTU RGB+D 60, NTU RGB+D 120, MSR-Daily 3D, N-UCLA, and HDM05) demonstrate that the proposed DS-SGCN framework achieves encouraging performance on the skeleton-based action recognition task.

Asymmetric information-regularized learning for skeleton-based action recognition

Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition

Graph-Temporal LSTM Networks for Skeleton-Based Action Recognition

An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

Richly Activated Graph Convolutional Network for Robust Skeleton-Based Action Recognition

Temporal Refinement Graph Convolutional Network for Skeleton-based Action Recognition

Recurrent Graph Convolutional Autoencoder for Unsupervised Skeleton-Based Action Recognition.

GAS-GCN: Gated Action-Specific Graph Convolutional Networks for Skeleton-Based Action Recognition.

Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks

Multi-Stage Attention-Enhanced Sparse Graph Convolutional Network for Skeleton-Based Action Recognition

Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognition

DSDC-GCN: Decoupled Static-Dynamic Co-occurrence Graph Convolutional Networks for Skeleton-Based Action Recognition

Self-Relational Graph Convolution Network for Skeleton-Based Action Recognition

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Multi‐temporal scale aggregation refinement graph convolutional network for skeleton‐based action recognition

Dual-Stream Structured Graph Convolution Network for Skeleton-Based Action Recognition

An improved spatial temporal graph convolutional network for robust skeleton-based action recognition

Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition.

A Tri-Attention Enhanced Graph Convolutional Network for Skeleton-Based Action Recognition

Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition

Human Skeleton Feature Optimizer and Adaptive Structure Enhancement Graph Convolution Network for Action Recognition