Abstract:It's common for current methods in skeleton-based action recognition to mainly consider capturing long-term temporal dependencies as skeleton sequences are typically long (>128 frames), which forms a challenging problem for previous approaches. In such conditions, short-term dependencies are few formally considered, which are critical for classifying similar actions. Most current approaches are consisted of interleaving spatial-only modules and temporal-only modules, where direct information flow among joints in adjacent frames are hindered, thus inferior to capture short-term motion and distinguish similar action pairs. To handle this limitation, we propose a general framework, coined as STGAT, to model cross-spacetime information flow. It equips the spatial-only modules with spatial-temporal modeling for regional perception. While STGAT is theoretically effective for spatial-temporal modeling, we propose three simple modules to reduce local spatial-temporal feature redundancy and further release the potential of STGAT, which (1) narrow the scope of self-attention mechanism, (2) dynamically weight joints along temporal dimension, and (3) separate subtle motion from static features, respectively. As a robust feature extractor, STGAT generalizes better upon classifying similar actions than previous methods, witnessed by both qualitative and quantitative results. STGAT achieves state-of-the-art performance on three large-scale datasets: NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400. Code is released.

Explorations of Skeleton Features for LSTM-based Action Recognition

Fusing Geometric Features for Skeleton-Based Action Recognition Using Multilayer LSTM Networks

An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

Relational Network for Skeleton-Based Action Recognition

An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data

Action Recognition Based on Global Optimal Similarity Measuring

Skeleton-based Action Recognition Using LSTM and CNN

Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks

Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks

Investigation of Different Skeleton Features for CNN-based 3D Action Recognition

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

Skeleton-based action recognition with hierarchical spatial reasoning and temporal stack learning network

Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks

Spatial Temporal Graph Attention Network for Skeleton-Based Action Recognition

Skeleton-based action recognition by part-aware graph convolutional networks

Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning

Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition

Fusing Higher-Order Features in Graph Neural Networks for Skeleton-Based Action Recognition

Hierarchical recurrent neural network for skeleton based action recognition

Generic Enhanced Ensemble Learning with Multi-Level Kinematic Constraints for 3D Action Recognition

Skeleton-Indexed Deep Multi-Modal Feature Learning for High Performance Human Action Recognition