Abstract:In this paper, we present a method (Action-Fusion) for human action recognition from depth maps and posture data using convolutional neural networks (CNNs). Two input descriptors are used for action representation. The first input is a depth motion image that accumulates consecutive depth maps of a human action, whilst the second input is a proposed moving joints descriptor which represents the motion of body joints over time. In order to maximize feature extraction for accurate action classification, three CNN channels are trained with different inputs. The first channel is trained with depth motion images (DMIs), the second channel is trained with both DMIs and moving joint descriptors together, and the third channel is trained with moving joint descriptors only. The action predictions generated from the three CNN channels are fused together for the final action classification. We propose several fusion score operations to maximize the score of the right action. The experiments show that the results of fusing the output of three channels are better than using one channel or fusing two channels only. Our proposed method was evaluated on three public datasets: 1) Microsoft action 3-D dataset (MSRAction3D); 2) University of Texas at Dallas-multimodal human action dataset; and 3) multimodal action dataset (MAD) dataset. The testing results indicate that the proposed approach outperforms most of existing state-of-the-art methods, such as histogram of oriented 4-D normals and Actionlet on MSRAction3D. Although MAD dataset contains a high number of actions (35 actions) compared to existing action RGB-D datasets, this paper surpasses a state-of-the-art method on the dataset by 6.84%.

A Combination Model of Shifting Joint Angle Changes With 3D-Deep Convolutional Neural Network to Recognize Human Activity

Human activity classification using deep learning based on 3D motion feature

Action Recognition with Joints-Pooled 3D Deep Convolutional Descriptors

Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition

3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks

Dynamic Action Recognition: A convolutional neural network model for temporally organized joint location data

Motion Recognition Based on Deep Learning and Human Joint Points

Human Joint Angle Estimation Using Deep Learning-Based Three-Dimensional Human Pose Estimation for Application in a Real Environment

Learning Dictionaries Of Sparse Codes Of 3d Movements Of Body Joints For Real-Time Human Activity Understanding

Deep Convolutional Neural Networks for Human Action Recognition Using Depth Maps and Postures

Action Recognition Using Interrelationships of 3D Joints and Frames Based on Angle Sine Relation and Distance Features Using Interrelationships

Hybrid convolution neural network with channel attention mechanism for sensor-based human activity recognition

Effective Human Action Recognition Using Global and Local Offsets of Skeleton Joints.

Infusing a Convolutional Neural Network with Encoded Joint Node Image Data to Recognize 25 Daily Human Activities.

Learning to Recognize 3D Human Action from A New Skeleton-based Representation Using Deep Convolutional Neural Networks

Human Action Recognition Based on Kinematic Similarity in Real Time.

A Novel Hybrid Deep Learning Model for Human Activity Recognition Based on Transitional Activities

Dynamic Edge Convolutional Neural Network for Skeleton-Based Human Action Recognition

Human activity recognition using deep learning approaches and single frame cnn and convolutional lstm

Action Recognition Based on Joint Trajectory Maps with Convolutional Neural Networks

A Novel Human Activity Recognition Method Using Joint Points Information