Abstract:Currently, human action recognition has witnessed remarkable progress, and its achievements have been applied to daily life. However, most methods extract features from only a single view within each modality, which may not comprehensively capture the diversity and complexity of actions. Moreover, the ineffective removal of redundant information can result in an inconspicuous description of key information. These issues cloud affect the final action recognition accuracy. To address these issues, this paper proposes a novel method for single-subject routine action recognition, which combines multi-view key information representation and multi-modal fusion. Firstly, the energy of non-primary motion areas is reduced by motion mean normalization in the depth video sequence, thereby enhancing key information of action. Then, depth motion history map (DMHM) and depth spatio-temporal energy map (DSTEM) are extracted from planes and axes, respectively. The proposed DMHM effectively preserves the spatio-temporal information of actions, DSTEM preserves the motion contour and energy information. In terms of skeleton sequences, statistical features and motion contribution degree of each joint are extracted from the view of motion distribution and weights, respectively. Finally, depth and skeleton features are fused to achieve multi-modal fusion-based action recognition. The proposed method highlights the information of the main motion areas, and achieves recognition accuracies of 96.70 on MSR-Action3D, 93.26 on UTD-MHAD, and above 97.73 on all tests of CZU-MHAD. The experimental results demonstrate that the proposed method effectively preserves action information and has better recognition accuracy than most existing methods.

Fusing Shape and Motion Matrices for View Invariant Action Recognition Using 3D Skeletons

Action Recognition Based on Global Optimal Similarity Measuring

Shifting Perspective to See Difference: A Novel Multi-View Method for Skeleton Based Action Recognition

Online Robust Action Recognition Based on a Hierarchical Model

3D Action Recognition Using Multi-Temporal Skeleton Visualization.

Fusing Geometric Features for Skeleton-Based Action Recognition Using Multilayer LSTM Networks

Action Recognition from Arbitrary Views Using 3D-Key-pose Set

Combining Adaptive Hierarchical Depth Motion Maps with Skeletal Joints for Human Action Recognition

View-invariant Human Action Recognition Via Robust Locally Adaptive Multi-View Learning

Using the Representation of 3D Skeleton Snippet for Human Action Recognition

A New Representation of Skeleton Sequences for 3D Action Recognition

Learning Informative Pairwise Joints with Energy-Based Temporal Pyramid for 3D Action Recognition

An effective representation for action recognition with human skeleton joints

Multi-view key information representation and multi-modal fusion for single-subject routine action recognition

Skeletal Quads: Human Action Recognition Using Joint Quadruples

Enhanced Skeleton Visualization for View Invariant Human Action Recognition.

Spatio–Temporal Image Representation of 3D Skeletal Movements for View-Invariant Action Recognition with Deep Convolutional Neural Networks

3D Action Recognition Using Data Visualization and Convolutional Neural Networks.

View Invariant Human Action Recognition Based on Factorization and HMMs.

Learning Spatial-Preserved Skeleton Representations for Few-Shot Action Recognition.

Improving Small-Scale Human Action Recognition Performance Using a 3D Heatmap Volume