Abstract:Action recognition is essential for many human-centered applications in the Internet of Things (IoT). Especially, in the Internet of Medical Things (IoMT), action recognition shows great importance in surgical assistance, patient monitoring, etc. Recently, 3-D skeleton sequence-based action recognition draws broad attention. It is a challenging task that needs effective modeling on intraframe skeleton representations and interframe temporal dynamics. Standard long short-term memory (LSTM)-based models are widely used for sequence modeling due to its long-term memory, yet they are unable to fully model the relationship between different body joints or persons to extract crucial co-occurrence features from different levels. To handle this shortcoming, we propose an attention-based multilevel co-occurrence graph convolutional LSTM (AMCGC-LSTM). By integrating graph convolutional networks (GCNs) into LSTM, the proposed model is capable of leveraging body structural information from skeletons and strengthening the multilevel co-occurrence (MC) feature learning. Specifically, we first design the spatial attention module for feature enhancement of key joints from skeleton inputs. Second, we design MC memory units coupled with GCN to automatically model the spatial relationship between joints, and simultaneously capture the co-occurrence features from different joints, persons, and frames. Finally, we construct aggregated features of MCs (AFMCs) from MC memory units to better represent the intraframe action context encoding, and leverage a concurrent LSTM (Co-LSTM) to further model their temporal dynamics for action recognition. Our model significantly outperforms mainstream methods on NTU RGB+D 60/120 data set, mutual action subset of NTU RGB+D 60/120 data set, and Northewestern-UCLA data set.

Action recognition using three dimension convolution and long short term memory

Short-Term Action Recognition by 3D Convolutional Neural Network with Pixel-Wise Evidences

Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition

3D Convolutional Neural Network for Action Recognition.

DC3D: A Video Action Recognition Network Based on Dense Connection

Human Action Recognition From Digital Videos Based on Deep Learning.

Temporal Pyramid Pooling-Based Convolutional Neural Network for Action Recognition

Beyond Frame-level CNN: Saliency-Aware 3-D CNN with LSTM for Video Action Recognition.

Lattice Long Short-Term Memory for Human Action Recognition

Attention-Based Multilevel Co-Occurrence Graph Convolutional LSTM for 3-D Action Recognition

3D-TDC: A 3D temporal dilation convolution framework for video action recognition

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Human Action Recognition Combining Sequential Dynamic Images and Two-Stream Convolutional Network

Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognition

Action recognition method based on a novel keyframe extraction method and enhanced 3D convolutional neural network

Enhanced Action Recognition With Visual Attribute-Augmented 3D Convolutional Neural Network

Visual Attribute-augmented Three-dimensional Convolutional Neural Network for Enhanced Human Action Recognition.

Two-Stream 3-D Convnet Fusion for Action Recognition in Videos with Arbitrary Size and Length

An Enhanced 3dcnn-Convlstm For Spatiotemporal Multimedia Data Analysis

Skeleton-based Action Recognition Using LSTM and CNN