Abstract:Action recognition is essential for many human-centered applications in the Internet of Things (IoT). Especially, in the Internet of Medical Things (IoMT), action recognition shows great importance in surgical assistance, patient monitoring, etc. Recently, 3-D skeleton sequence-based action recognition draws broad attention. It is a challenging task that needs effective modeling on intraframe skeleton representations and interframe temporal dynamics. Standard long short-term memory (LSTM)-based models are widely used for sequence modeling due to its long-term memory, yet they are unable to fully model the relationship between different body joints or persons to extract crucial co-occurrence features from different levels. To handle this shortcoming, we propose an attention-based multilevel co-occurrence graph convolutional LSTM (AMCGC-LSTM). By integrating graph convolutional networks (GCNs) into LSTM, the proposed model is capable of leveraging body structural information from skeletons and strengthening the multilevel co-occurrence (MC) feature learning. Specifically, we first design the spatial attention module for feature enhancement of key joints from skeleton inputs. Second, we design MC memory units coupled with GCN to automatically model the spatial relationship between joints, and simultaneously capture the co-occurrence features from different joints, persons, and frames. Finally, we construct aggregated features of MCs (AFMCs) from MC memory units to better represent the intraframe action context encoding, and leverage a concurrent LSTM (Co-LSTM) to further model their temporal dynamics for action recognition. Our model significantly outperforms mainstream methods on NTU RGB+D 60/120 data set, mutual action subset of NTU RGB+D 60/120 data set, and Northewestern-UCLA data set.

Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification

Prototype Learning for Micro-gesture Classification

Micro-gesture Online Recognition using Learnable Query Points

MVHANet: Multi-view Hierarchical Aggregation Network for Skeleton-Based Hand Gesture Recognition

Surface Electromyography-based Gesture Recognition by Multi-view Deep Learning.

Semg-Based Gesture Recognition with Embedded Virtual Hand Poses and Adversarial Learning.

Fusing Geometric Features for Skeleton-Based Action Recognition Using Multilayer LSTM Networks

Gesture Classification in Electromyography Signals for Real-Time Prosthetic Hand Control Using a Convolutional Neural Network-Enhanced Channel Attention Model

Integration of Convolutional Neural Network and Vision Transformer for Gesture Recognition Using Semg

Micro-expression recognition using 3D DenseNet fused Squeeze-and-Excitation Networks

Real-Time Monocular Skeleton-Based Hand Gesture Recognition Using 3D-Jointsformer

Gesture recognition based on deep deformable 3D convolutional neural networks

Deep self-supervised learning with visualisation for automatic gesture recognition

JGULF: Joint global and unilateral local feature network for micro-expression recognition

Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer

Learning a compact embedding for fine-grained few-shot static gesture recognition

Attention-based hand semantic segmentation and gesture recognition using deep networks

Hand Gesture Recognition using Deep Feature Fusion Network based on Wearable Sensors

Multi-View Fusion Network-Based Gesture Recognition Using sEMG Data

MFA-Net: Motion Feature Augmented Network for Dynamic Hand Gesture Recognition from Skeletal Data

Attention-Based Multilevel Co-Occurrence Graph Convolutional LSTM for 3-D Action Recognition