Abstract:The overall pipeline of our proposed method. The skeleton data is first input into RGCN to obtain basic feature expressions. RGCN can learn more spatial motion information of actions. Features with different temporal resolutions are then modulated in the temporal and spatial dimensions and aggregated into features with rich discriminative temporal information for final classification. Skeleton‐based human action recognition is gaining significant attention and finding widespread application in various fields, such as virtual reality and human‐computer interaction systems. Recent studies have highlighted the effectiveness of graph convolutional network (GCN) based methods in this task, leading to a remarkable improvement in prediction accuracy. However, most GCN‐based methods overlook the varying contributions of self, centripetal and centrifugal subsets. Besides, only a single‐scale temporal feature is adopted, and the multi‐temporal scale information is ignored. To this end, firstly, in order to differentiate the importance of different skeleton subsets, we develop a refinement graph convolution, which can adaptively learn a weight for each subset feature. Secondly, a multi‐temporal scale aggregation module is proposed to extract more discriminative temporal dynamic information. Furthermore, a multi‐temporal scale aggregation refinement graph convolutional network (MTSA‐RGCN) is proposed, and four‐stream structure is also adopted in this paper, which can comprehensively model complementary features and eventually achieves a significant performance boost. In the empirical experiments, the performance of our approach has been greatly improved on both NTU‐RGB+D 60 and NTU‐RGB+D 120 datasets, compared to other state‐of‐the‐art methods.

Traffic police command gesture recognition technology based on machine vision and two-stream spatio-temporal attention graph convolutional network

Chinese Traffic Police Gesture Recognition Based on Graph Convolutional Network in Natural Scene

Traffic Police 3D Gesture Recognition Based on Spatial–Temporal Fully Adaptive Graph Convolutional Network

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Attention Mechanism Based on Improved Spatial-Temporal Convolutional Neural Networks for Traffic Police Gesture Recognition

Visual Recognition of traffic police gestures with convolutional pose machine and handcrafted features

Low light recognition of traffic police gestures based on lightweight extraction of skeleton features

A Two-stream Neural Network for Pose-based Hand Gesture Recognition

Multi-Stage Attention-Enhanced Sparse Graph Convolutional Network for Skeleton-Based Action Recognition

Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks

Traffic Police Gesture Command Recognition Based on Convolutional Neural Network AlexNet

An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

Spatial Temporal Graph Attention Network for Skeleton-Based Action Recognition

Multi-Scale Adaptive Aggregate Graph Convolutional Network for Skeleton-Based Action Recognition

Pose-Guided Graph Convolutional Networks for Skeleton-Based Action Recognition

Multi-stream P&U adaptive graph convolutional networks for skeleton-based action recognition

Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition

Multi‐temporal scale aggregation refinement graph convolutional network for skeleton‐based action recognition

A Tri-Attention Enhanced Graph Convolutional Network for Skeleton-Based Action Recognition

Simple But Effective: Upper-Body Geometric Features for Traffic Command Gesture Recognition