Abstract:Over the past few years, automatic recognition of human interactions has drawn significant attention from researchers working in the field of Artificial Intelligence (AI). And feature extraction is one of the most critical tasks in developing efficient Human Interaction Recognition (HIR) systems. Moreover, recent researches in computer vision suggest that robust features lead to higher recognition accuracies. Hence, an improved HIR system has been proposed in this paper that combines 2D and 3D features extracted using machine learning and deep learning techniques. These discriminative features result in accurate classification and help avoid misclassification of similar interactions. Ten keyframes have been extracted from each video to reduce computational complexity. Next, these frames have been preprocessed using image normalization and noise removal techniques. The Region Of Interest (ROI), which contains the two humans involved in the interaction, has been extracted using motion detection. Then, the human silhouettes have been segmented using the GrabCut algorithm. Next, the extracted silhouettes have been converted into 3D meshes and their heat kernel signatures (HKS) have been obtained to extract key body points. A Convolutional Neural Network (CNN) has been used to extract full-body features from 2D full-body silhouettes. Then, topological and geometric features have been extracted from the key body points. Finally, the combined feature vector has been fed into Long Short-Term Memory (LSTM) and each interaction has been recognized using a Softmax classifier. The proposed system has been validated via extensive experimentation on three challenging RGB+D datasets. The recognition accuracies of 91.63%, 90.54%, and 90.13% have been achieved with the SBU Kinect Interaction, NTU RGB+D, and ISR-UoL 3D social activity datasets respectively. The results of extensive experiments performed on the proposed system suggest that it can be used effectively for various applications, such as security, surveillance, health monitoring, and assisted living.

A Vision-Based Measure of Environmental Effects on Inferring Human Intention During Human Robot Interaction

Better Than Humans: a Method for Inferring Consumer Shopping Intentions by Reading Facial Expressions

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

Bayesian Intention for Enhanced Human Robot Collaboration

Benchmarking Adaptive Intelligence and Computer Vision on Human-Robot Collaboration

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Cross-View Human Intention Recognition for Human-Robot Collaboration

Virtual data generation for human intention prediction based on digital modeling of human-robot collaboration

Human-Robot Emotional Interaction Model Based on Reinforcement Learning

Estimating Emotional Intensity from Body Poses for Human-Robot Interaction

Implementation of Engagement Detection for Human–Robot Interaction in Complex Environments

Emotion-Age-Gender-Nationality Based Intention Understanding in Human–Robot Interaction Using Two-Layer Fuzzy Support Vector Regression

A Practical, Robust, Accurate Gaze-Based Intention Inference Method for Everyday Human-Robot Interaction

A Dynamic Head Gesture Recognition Method for Real-time Intention Inference and Its Application to Visual Human-robot Interaction

Hybrid Recurrent Neural Network Architecture-Based Intention Recognition for Human-Robot Collaboration

An LSTM-Based Approach for Understanding Human Interactions Using Hybrid Feature Descriptors Over Depth Sensors

A Vision-based Human Digital Twin Modelling Approach for Adaptive Human-Robot Collaboration

Where and Why Are They Looking? Jointly Inferring Human Attention and Intentions in Complex Tasks

Inferring Human Intent from Video by Sampling Hierarchical Plans

Towards Inferring Users' Impressions of Robot Performance in Navigation Scenarios

Learning Multimodal Latent Dynamics for Human-Robot Interaction