Abstract:Action recognition techniques based on skeleton data are receiving more and more attention in the field of computer vision due to their ability to adapt to dynamic environments and complex backgrounds. Topologizing human skeleton data as spatial-temporal graphs and processing them using graph convolutional networks (GCNs) has been shown to produce good recognition results. However, with existing GCN methods, a fixed-size convolution kernel is often used to extract time-domain features, which may not be very suitable for multi-level model structures. Equal proportion fusion of different streams in a multi-stream network may ignore the difference in recognition ability of different streams, and these will affect the final recognition result. In this paper, we are proposing (1) a multi-scale dilated temporal graph convolution layer (MDTGCL) and (2) a multi-branch feature fusion (MFF) structure. The MDTGCL utilizes multiple convolution kernels and dilated convolution to better adapt to the multi-layer structure of the GCN model and to obtain longer periods of contextual spatial-temporal information, resulting in richer behavioural features. MFF entails weighted fusion based on the results of multi-stream outputs, and this is used to obtain the final recognition results. As higher-order skeleton data are highly discriminative and more conducive to human action recognition, we used spatial information on joints and bones and their multiple motion, as well as angle information pertaining to bones, to model together in this study. By combining the above, we designed a multi-stream, multi-scale dilated spatial-temporal graph convolutional network (2M-STGCN) model and conducted extensive experiments with two large datasets (NTU RGB+D 60 and Kinetics Skeleton 400), which showed that our model performs at SOTA level.

Symmetrical Enhanced Fusion Network for Skeleton-Based Action Recognition

A Spatiotemporal Fusion Network for Skeleton-Based Action Recognition

Multi-Modality Adaptive Feature Fusion Graph Convolutional Network for Skeleton-Based Action Recognition

Hybrid Features for Skeleton-Based Action Recognition Based on Network Fusion.

Skeleton-based Action Recognition with Multi-Stream, Multi-Scale Dilated Spatial-Temporal Graph Convolution Network

Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognition

Joints-Centered Spatial-Temporal Features Fused Skeleton Convolution Network for Action Recognition

Fusing Higher-Order Features in Graph Neural Networks for Skeleton-Based Action Recognition

Skeleton-based Action Recognition with Multi-stream, Temporal-Channel Enhanced Graph Convolution Network

Temporal Enhanced Multi-Stream Graph Convolutional Nerual Networks For Skeleton-Based Action Recognition

Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks

Lightweight Skeleton-Based Action Recognition Model Based on Global–local Feature Extraction and Fusion

Dual-Stream Structured Graph Convolution Network for Skeleton-Based Action Recognition

Self-Relational Graph Convolution Network for Skeleton-Based Action Recognition

Adaptive Spatiotemporal Graph Convolutional Network with Intermediate Aggregation of Multi-Stream Skeleton Features for Action Recognition

Multi-Stage Attention-Enhanced Sparse Graph Convolutional Network for Skeleton-Based Action Recognition

Skeleton Feature Fusion Based on Multi-Stream LSTM for Action Recognition.

A Tri-Attention Enhanced Graph Convolutional Network for Skeleton-Based Action Recognition

Multi‐temporal scale aggregation refinement graph convolutional network for skeleton‐based action recognition

Behavioral Recognition of Skeletal Data Based on Targeted Dual Fusion Strategy

Skeleton-Based Action Recognition with Focusing-Diffusion Graph Convolutional Networks