Abstract:Action recognition based on 3D skeleton sequences has gained considerable attention in recent years. Due to effectively representing the spatial and the temporal characters of skeleton sequences, the Covariance Matrix (CM) features combined with the Long Short-Term Memory (LSTM) network is an effective and reasonable roadmap to enhance the action recognition accuracy. However, the CM features in the existing recognition models are computed from the raw data without normalization or with static normalization. Moreover, a CM feature is calculated from all coordinates in one frame, treating all coordinates in three axes identically and neglecting the relationship of the coordinates in the same axe. In this paper, an end to end deep learning framework is proposed that includes a normalization layer dynamically adapting to data distribution and inference procedure. After normalization, the three covariance feature sequences from the coordinates in three axes are produced from the sliding windows and are fused into one fusion matrix using a convolution layer. Finally, the fusion matrix is sequentially fed into an LSTM network to recognize skeleton action. The novelty of the proposed framework is combining the adaptive preprocessing and the features fusion to the LSTM network and improving the recognition accuracy by optimizing the quality of the features rather than network construction. In the experiments, the proposed framework is verified on the public datasets and one student action dataset collected from a real classroom. The experimental results demonstrate that the proposed method achieves a significant improvement in accuracy compared to the state-of-the-art methods. It can be concluded that the proposed framework can not only accurately capture the correlation of joints in the same frame but can also effectively express the dependences of sequential frames.

An optimization method of human skeleton keyframes selection for action recognition

Improved Key Poses Model for Skeleton-Based Action Recognition.

Human action recognition method based on key frame and skeleton information*

Shifting Perspective to See Difference: A Novel Multi-View Method for Skeleton Based Action Recognition

Action Recognition Based on Global Optimal Similarity Measuring

View-independent Representation with Frame Interpolation Method for Skeleton-Based Human Action Recognition

Combine Multi-order Representation Learning and Frame Optimization Learning for Skeleton-based Action Recognition

Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformation

Effective Human Action Recognition Using Global and Local Offsets of Skeleton Joints.

Key Joints Selection and Spatiotemporal Mining for Skeleton-Based Action Recognition

Improving bag-of-poses with semi-temporal pose descriptors for skeleton-based action recognition

An effective representation for action recognition with human skeleton joints

MKTZ: multi-semantic embedding and key frame masking techniques for zero-shot skeleton action recognition

Action Recognition and Prediction of Human Skeleton Based on BSCPs-RF

Full-Dimensional Optimizable Network: A Channel, Frame and Joint-Specific Network Modeling for Skeleton-Based Action Recognition

Human Action Recognition Using Key-Frame Attention-Based LSTM Networks

Skeleton-based human action recognition using multiple sequence alignment

Optimizing Features Quality: A Normalized Covariance Fusion Framework for Skeleton Action Recognition.

Human Action Recognition Based on Key Frames

Optimization-based key frame extraction for motion capture animation

Action Recognition Based on 3D Skeleton and RGB Frame Fusion