Abstract:Action recognition is an enabling technology for many real world applications, such as human-computer interaction, surveillance, video retrieval, retirement home monitoring, and robotics. In the past decade, it has attracted a great amount of interest in the research community. Recently, the commoditization of depth sensors has generated much excitement in action recognition from depth sensors. New depth sensor technology has enabled many applications that were not feasible before. On one hand, action recognition becomes far easier with depth sensors. On the other hand, the drive to recognize more complex actions presents new challenges. One crucial aspect of action recognition is to extract discriminative features. The depth maps have completely different characteristics from the RGB images. Directly applying features designed for RGB images does not work. Complex actions usually involve complicated temporal structures, human-object interactions, and person-person contacts. New machine learning algorithms need to be developed to learn these complex structures. This work enables the reader to quickly familiarize themselves with the latest research in depth-sensor based action recognition, and to gain a deeper understanding of recently developed techniques. It will be of great use for both researchers and practitioners who are interested in human action recognition with depth sensors. The text focuses on feature representation and machine learning algorithms for action recognition from depth sensors. After presenting a comprehensive overview of the state of the art in action recognition from depth data, the authors then provide in-depth descriptions of their recently developed feature representations and machine learning techniques, including lower-level depth and skeleton features, higher-level representations to model the temporal structure and human-object interactions, and feature selection techniques for occlusion handling.

Discriminative Feature Learning with Constraints of Category and Temporal for Action Recognition.

Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition

Deep spatiotemporal LSTM network with temporal pattern feature for 3D human action recognition

Temporal Distinct Representation Learning for Action Recognition

Discriminative Multi-View Subspace Feature Learning for Action Recognition

Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition

3D Action Recognition Using Multi-Temporal Depth Motion Maps and Fisher Vector

Human Action Recognition with Depth Cameras

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Action Recognition from Depth Sequences Using Weighted Fusion of 2D and 3D Auto-Correlation of Gradients Features

Short-Term Action Recognition by 3D Convolutional Neural Network with Pixel-Wise Evidences

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Collaborative Spatio-temporal Feature Learning for Video Action Recognition

Online Robust Action Recognition Based on a Hierarchical Model

Spatiotemporal Multimodal Learning With 3D CNNs for Video Action Recognition

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Deep Convolutional Neural Networks for Action Recognition Using Depth Map Sequences

Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection

DB-LSTM: Densely-connected Bi-directional LSTM for Human Action Recognition

Human action recognition using Adaptive Hierarchical Depth Motion Maps and Gabor filter

Dynamic Spatio-Temporal Specialization Learning for Fine-Grained Action Recognition