Abstract:Human Activity Recognition (HAR) is a key building block of many emerging applications such as intelligent mobility, sports analytics, ambient-assisted living and human-robot interaction. With robust HAR, systems will become more human-aware, leading towards much safer and empathetic autonomous systems. While human pose detection has made significant progress with the dawn of deep convolutional neural networks (CNNs), the state-of-the-art research has almost exclusively focused on a single sensing modality, especially video. However, in safety critical applications it is imperative to utilize multiple sensor modalities for robust operation. To exploit the benefits of state-of-the-art machine learning techniques for HAR, it is extremely important to have multimodal datasets. In this paper, we present a novel, multi-modal sensor dataset that encompasses nine indoor activities, performed by 16 participants, and captured by four types of sensors that are commonly used in indoor applications and autonomous vehicles. This multimodal dataset is the first of its kind to be made openly available and can be exploited for many applications that require HAR, including sports analytics, healthcare assistance and indoor intelligent mobility. We propose a novel data preprocessing algorithm to enable adaptive feature extraction from the dataset to be utilized by different machine learning algorithms. Through rigorous experimental evaluations, this paper reviews the performance of machine learning approaches to posture recognition, and analyses the robustness of the algorithms. When performing HAR with the RGB-Depth data from our new dataset, machine learning algorithms such as a deep neural network reached a mean accuracy of up to 96.8% for classification across all stationary and dynamic activities

Supervised Learning for Human Action Recognition from Multiple Kinects

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Using a Selective Ensemble Support Vector Machine to Fuse Multimodal Features for Human Action Recognition

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Analysis and Evaluation of Kinect-based Action Recognition Algorithms

Online Robust Action Recognition Based on a Hierarchical Model

Human Action Recognition Using Deep Learning Methods.

View-invariant Human Action Recognition Via Robust Locally Adaptive Multi-View Learning

Indoor Human Action Recognition Based on Dual Kinect V2 and Improved Ensemble Learning Method

Exploiting deep residual networks for human action recognition from skeletal data

Learning and Recognizing Human Action from Skeleton Movement with Deep Residual Neural Networks

Human Action Recognition of Spatiotemporal Parameters for Skeleton Sequences Using MTLN Feature Learning Framework

A multimodal approach for human activity recognition based on skeleton and RGB data

Adaptive Feature Processing for Robust Human Activity Recognition on a Novel Multi-Modal Dataset

Deep learning-based multi-view 3D-human action recognition using skeleton and depth data

Multi-view Multi-modal Approach Based on 5S-CNN and BiLSTM Using Skeleton, Depth and RGB Data for Human Activity Recognition

RGB-D based human action recognition using evolutionary self-adaptive extreme learning machine with knowledge-based control parameters

Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation

Learning to Recognize 3D Human Action from A New Skeleton-based Representation Using Deep Convolutional Neural Networks

Human action recognition based on convolutional neural network

Semi-Supervised Multiple Feature Analysis for Action Recognition