Abstract:Over the past few years, automatic recognition of human interactions has drawn significant attention from researchers working in the field of Artificial Intelligence (AI). And feature extraction is one of the most critical tasks in developing efficient Human Interaction Recognition (HIR) systems. Moreover, recent researches in computer vision suggest that robust features lead to higher recognition accuracies. Hence, an improved HIR system has been proposed in this paper that combines 2D and 3D features extracted using machine learning and deep learning techniques. These discriminative features result in accurate classification and help avoid misclassification of similar interactions. Ten keyframes have been extracted from each video to reduce computational complexity. Next, these frames have been preprocessed using image normalization and noise removal techniques. The Region Of Interest (ROI), which contains the two humans involved in the interaction, has been extracted using motion detection. Then, the human silhouettes have been segmented using the GrabCut algorithm. Next, the extracted silhouettes have been converted into 3D meshes and their heat kernel signatures (HKS) have been obtained to extract key body points. A Convolutional Neural Network (CNN) has been used to extract full-body features from 2D full-body silhouettes. Then, topological and geometric features have been extracted from the key body points. Finally, the combined feature vector has been fed into Long Short-Term Memory (LSTM) and each interaction has been recognized using a Softmax classifier. The proposed system has been validated via extensive experimentation on three challenging RGB+D datasets. The recognition accuracies of 91.63%, 90.54%, and 90.13% have been achieved with the SBU Kinect Interaction, NTU RGB+D, and ISR-UoL 3D social activity datasets respectively. The results of extensive experiments performed on the proposed system suggest that it can be used effectively for various applications, such as security, surveillance, health monitoring, and assisted living.

Deep Learning-based human activity recognition using RGB images in Human-robot collaboration

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

Deep Learning Methods for Human Behavior Recognition.

Human Action Recognition Using Deep Learning Methods.

Human Action Recognition From Digital Videos Based on Deep Learning.

A Vision-Based Measure of Environmental Effects on Inferring Human Intention During Human Robot Interaction

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Human-Robot Collaboration by Intention Recognition Using Deep LSTM Neural Network

An LSTM-Based Approach for Understanding Human Interactions Using Hybrid Feature Descriptors Over Depth Sensors

Modelling Human Body Pose for Action Recognition Using Deep Neural Networks

Deep Learning-Based Human Action Recognition in Videos

Deep Ensemble Learning for Human Action Recognition in Still Images

Hybrid Recurrent Neural Network Architecture-Based Intention Recognition for Human-Robot Collaboration

Prediction of Intentions Behind a Single Human Action: an Application of Convolutional Neural Network

LSTM-Based Neural Network to Recognize Human Activities Using Deep Learning Techniques

Deep Learning-Based Human Motion Recognition for Predictive Context-Aware Human-Robot Collaboration

Exploiting deep residual networks for human action recognition from skeletal data

Towards a Safe Human-Robot Collaboration Using Information on Human Worker Activity

Human Action Recognition by Learning Spatio-Temporal Features with Deep Neural Networks

Learning and Recognizing Human Action from Skeleton Movement with Deep Residual Neural Networks

Robot Awareness of Human Behaviors and Interacting with Humans by A Two-Step Identifying Approach combining LSTM and SSD.