Attentional weighting strategy-based dynamic GCN for skeleton-based action recognition

Kai Hu,Junlan Jin,Chaowen Shen,Min Xia,Liguo Weng

DOI: https://doi.org/10.1007/s00530-023-01082-1

IF: 3.9

2023-04-02

Multimedia Systems

Abstract:Graph Convolutional Networks (GCNs) have become the standard skeleton-based human action recognition research paradigm. As a core component in graph convolutional networks, the construction of graph topology often significantly impacts the accuracy of classification. Considering that the fixed physical graph topology cannot capture the non-physical connection relationship of the human body, existing methods capture more flexible node relationships by constructing dynamic graph structures. This paper proposes a novel attentional weighting strategy-based dynamic GCN (AWD-GCN). We construct a new dynamic adjacency matrix, which uses the attention weighting mechanism to simultaneously capture the dynamic relationships among the three partitions of the human skeleton under multiple actions to extract the discriminative action features fully. In addition, considering the importance of skeletal node position features for action differentiation, we propose new multi-scale position attention and multi-level attention. We use a multi-scale modelling method to capture the complex relationship between skeletal node position features, which is helpful in distinguishing human action in different spatial scales. Extensive experiments on two challenging datasets, NTU-RGB+D and Skeleton-Kinetics, demonstrate the effectiveness and superiority of our method.

computer science, information systems, theory & methods