Abstract:Action recognition techniques based on skeleton data are receiving more and more attention in the field of computer vision due to their ability to adapt to dynamic environments and complex backgrounds. Topologizing human skeleton data as spatial-temporal graphs and processing them using graph convolutional networks (GCNs) has been shown to produce good recognition results. However, with existing GCN methods, a fixed-size convolution kernel is often used to extract time-domain features, which may not be very suitable for multi-level model structures. Equal proportion fusion of different streams in a multi-stream network may ignore the difference in recognition ability of different streams, and these will affect the final recognition result. In this paper, we are proposing (1) a multi-scale dilated temporal graph convolution layer (MDTGCL) and (2) a multi-branch feature fusion (MFF) structure. The MDTGCL utilizes multiple convolution kernels and dilated convolution to better adapt to the multi-layer structure of the GCN model and to obtain longer periods of contextual spatial-temporal information, resulting in richer behavioural features. MFF entails weighted fusion based on the results of multi-stream outputs, and this is used to obtain the final recognition results. As higher-order skeleton data are highly discriminative and more conducive to human action recognition, we used spatial information on joints and bones and their multiple motion, as well as angle information pertaining to bones, to model together in this study. By combining the above, we designed a multi-stream, multi-scale dilated spatial-temporal graph convolutional network (2M-STGCN) model and conducted extensive experiments with two large datasets (NTU RGB+D 60 and Kinetics Skeleton 400), which showed that our model performs at SOTA level.

Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

Multi-Scale Spatial Temporal Graph Neural Network for Skeleton-Based Action Recognition

Multi-Scale Mixed Dense Graph Convolution Network for Skeleton-Based Action Recognition.

Multi-scale Gated Graph Convolutional Network for Skeleton-based Action Recognition

Multi-Stage Attention-Enhanced Sparse Graph Convolutional Network for Skeleton-Based Action Recognition

Multi-Scale Adaptive Aggregate Graph Convolutional Network for Skeleton-Based Action Recognition

Multi-Scale Adaptive Graph Convolution Network for Skeleton-Based Action Recognition

Graph-Temporal LSTM Networks for Skeleton-Based Action Recognition

Multi-Part Adaptive Graph Convolutional Network for Skeleton-Based Action Recognition.

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Skeleton-based Action Recognition with Multi-stream, Temporal-Channel Enhanced Graph Convolution Network

Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition.

Skeleton-based Action Recognition with Multi-Stream, Multi-Scale Dilated Spatial-Temporal Graph Convolution Network

Self-Relational Graph Convolution Network for Skeleton-Based Action Recognition

Multi-modality Dense Graph Convolution Network for Skeleton-Based Action Recognition

Lightweight Multi-Scale Spatiotemporal Graph Convolutional Network for Skeleton-Based Action Recognition

Multi-Scale Spatial-Temporal Self-Attention Graph Convolutional Networks for Skeleton-based Action Recognition

Multi‐temporal scale aggregation refinement graph convolutional network for skeleton‐based action recognition

Channel attention and multi-scale graph neural networks for skeleton-based action recognition

Multi-Branch Spatial-Temporal Attention Graph Convolution Network for Skeleton-based Action Recognition

Densely Connected and Multiple Temporal Graph Convolution Networks for Skeleton-based Action Recognition