Abstract:The computer vision community is currently focusing on solving action recognition problems in real videos, which contain thousands of samples with many challenges. In this process, Deep Convolutional Neural Networks (D-CNNs) have played a significant role in advancing the state-of-the-art in various vision-based action recognition systems. Recently, the introduction of residual connections in conjunction with a more traditional CNN model in a single architecture called Residual Network (ResNet) has shown impressive performance and great potential for image recognition tasks. In this paper, we investigate and apply deep ResNets for human action recognition using skeletal data provided by depth sensors. Firstly, the 3D coordinates of the human body joints carried in skeleton sequences are transformed into image-based representations and stored as RGB images. These color images are able to capture the spatial-temporal evolutions of 3D motions from skeleton sequences and can be efficiently learned by D-CNNs. We then propose a novel deep learning architecture based on ResNets to learn features from obtained color-based representations and classify them into action classes. The proposed method is evaluated on three challenging benchmark datasets including MSR Action 3D, KARD, and NTU-RGB+D datasets. Experimental results demonstrate that our method achieves state-of-the-art performance for all these benchmarks whilst requiring less computation resource. In particular, the proposed method surpasses previous approaches by a significant margin of 3.4% on MSR Action 3D dataset, 0.67% on KARD dataset, and 2.5% on NTU-RGB+D dataset.

3D Human Activity Recognition Using Skeletal Data from RGBD Sensors.

Human motion recognition using three-dimensional skeleton model based on RGBD vision system

An effective representation for action recognition with human skeleton joints

3d Human Action Recognition Based On The Spatial-Temporal Moving Skeleton Descriptor

Effective Human Action Recognition Using Global and Local Offsets of Skeleton Joints.

RGB-D Camera based Human Limb Movement Recognition and Tracking in Supine Positions.

Human Action Recognition Based on Kinematic Similarity in Real Time.

Online Robust Action Recognition Based on a Hierarchical Model

A multimodal approach for human activity recognition based on skeleton and RGB data

Human Action Recognition Based on Depth Images from Microsoft Kinect

Learning to Recognize 3D Human Action from A New Skeleton-based Representation Using Deep Convolutional Neural Networks

Study on Human Body Action Recognition.

Exploiting deep residual networks for human action recognition from skeletal data

Improved Key Poses Model for Skeleton-Based Action Recognition.

RGB-Depth Feature for 3D Human Activity Recognition

Using the Representation of 3D Skeleton Snippet for Human Action Recognition

Accurate And Real-Time Human Action Recognition Based On 3d Skeleton

Research for Human Action Recognition Based on Depth Information

3D Action Recognition Using Multi-Temporal Skeleton Visualization.

Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition

Action Recognition Based on 3D Skeleton and RGB Frame Fusion