Abstract:The computer vision community is currently focusing on solving action recognition problems in real videos, which contain thousands of samples with many challenges. In this process, Deep Convolutional Neural Networks (D-CNNs) have played a significant role in advancing the state-of-the-art in various vision-based action recognition systems. Recently, the introduction of residual connections in conjunction with a more traditional CNN model in a single architecture called Residual Network (ResNet) has shown impressive performance and great potential for image recognition tasks. In this paper, we investigate and apply deep ResNets for human action recognition using skeletal data provided by depth sensors. Firstly, the 3D coordinates of the human body joints carried in skeleton sequences are transformed into image-based representations and stored as RGB images. These color images are able to capture the spatial-temporal evolutions of 3D motions from skeleton sequences and can be efficiently learned by D-CNNs. We then propose a novel deep learning architecture based on ResNets to learn features from obtained color-based representations and classify them into action classes. The proposed method is evaluated on three challenging benchmark datasets including MSR Action 3D, KARD, and NTU-RGB+D datasets. Experimental results demonstrate that our method achieves state-of-the-art performance for all these benchmarks whilst requiring less computation resource. In particular, the proposed method surpasses previous approaches by a significant margin of 3.4% on MSR Action 3D dataset, 0.67% on KARD dataset, and 2.5% on NTU-RGB+D dataset.

Recognizing Human Actions by Using the Evolving Remote Supervised Method of Spiking Neural Networks.

Event-based Action Recognition Using Motion Information and Spiking Neural Networks

Human Tumble Action Recognition Using Spiking Neuron Network

Human Action Recognition Using Deep Learning Methods.

A Gesture Recognition Method Based On Spiking Neural Networks For Cognition Development

Human Action Recognition From Digital Videos Based on Deep Learning.

Recognizing Scoring in Basketball Game from AER Sequence by Spiking Neural Networks.

Learning and Recognizing Human Action from Skeleton Movement with Deep Residual Neural Networks

ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition

Human Action Recognition Based on Spiking Neural Networks

2D versus 3D Convolutional Spiking Neural Networks Trained with Unsupervised STDP for Human Action Recognition

Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning

Online Human Action Detection using Joint Classification-Regression Recurrent Neural Networks

Action Recognition Scheme Based on Skeleton Representation with DS-LSTM Network.

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

Skeleton-based action recognition with hierarchical spatial reasoning and temporal stack learning network

Learning to Recognize 3D Human Action from A New Skeleton-based Representation Using Deep Convolutional Neural Networks

Spatial-Temporal Neural Networks For Action Recognition

Exploiting deep residual networks for human action recognition from skeletal data

Spiking Two-Stream Methods with Unsupervised STDP-based Learning for Action Recognition

Deep Learning-Based Human Action Recognition in Videos