Abstract:Action recognition techniques based on skeleton data are receiving more and more attention in the field of computer vision due to their ability to adapt to dynamic environments and complex backgrounds. Topologizing human skeleton data as spatial-temporal graphs and processing them using graph convolutional networks (GCNs) has been shown to produce good recognition results. However, with existing GCN methods, a fixed-size convolution kernel is often used to extract time-domain features, which may not be very suitable for multi-level model structures. Equal proportion fusion of different streams in a multi-stream network may ignore the difference in recognition ability of different streams, and these will affect the final recognition result. In this paper, we are proposing (1) a multi-scale dilated temporal graph convolution layer (MDTGCL) and (2) a multi-branch feature fusion (MFF) structure. The MDTGCL utilizes multiple convolution kernels and dilated convolution to better adapt to the multi-layer structure of the GCN model and to obtain longer periods of contextual spatial-temporal information, resulting in richer behavioural features. MFF entails weighted fusion based on the results of multi-stream outputs, and this is used to obtain the final recognition results. As higher-order skeleton data are highly discriminative and more conducive to human action recognition, we used spatial information on joints and bones and their multiple motion, as well as angle information pertaining to bones, to model together in this study. By combining the above, we designed a multi-stream, multi-scale dilated spatial-temporal graph convolutional network (2M-STGCN) model and conducted extensive experiments with two large datasets (NTU RGB+D 60 and Kinetics Skeleton 400), which showed that our model performs at SOTA level.

Learning Local Part Motion Representation for Skeleton-based Action Recognition

Fusing Geometric Features for Skeleton-Based Action Recognition Using Multilayer LSTM Networks

Multi-source Learning for Skeleton -Based Action Recognition Using Deep LSTM Networks

Skeleton Feature Fusion Based on Multi-Stream LSTM for Action Recognition.

Explorations of Skeleton Features for LSTM-based Action Recognition

Joints-Centered Spatial-Temporal Features Fused Skeleton Convolution Network for Action Recognition

Graph-Temporal LSTM Networks for Skeleton-Based Action Recognition

Exploring a Rich Spatial-Temporal Dependent Relational Model for Skeleton-Based Action Recognition by Bidirectional LSTM-CNN.

LAGA-Net: Local-and-Global Attention Network for Skeleton Based Action Recognition

Hybrid Features for Skeleton-Based Action Recognition Based on Network Fusion.

Global Shapes and Salient Joints Features Learning for Skeleton-Based Action Recognition

Multisource Learning for Skeleton-Based Action Recognition Using Deep LSTM and CNN

Skeleton-Indexed Deep Multi-Modal Feature Learning for High Performance Human Action Recognition

Temporal Enhanced Multi-Stream Graph Convolutional Nerual Networks For Skeleton-Based Action Recognition

An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

Skeleton-based Action Recognition with Multi-Stream, Multi-Scale Dilated Spatial-Temporal Graph Convolution Network

Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognition

Skeleton-based Action Recognition Using LSTM and CNN

Multiple temporal scale aggregation graph convolutional network for skeleton-based action recognition

Skeleton-based action recognition by part-aware graph convolutional networks