Abstract:Over the past few years, automatic recognition of human interactions has drawn significant attention from researchers working in the field of Artificial Intelligence (AI). And feature extraction is one of the most critical tasks in developing efficient Human Interaction Recognition (HIR) systems. Moreover, recent researches in computer vision suggest that robust features lead to higher recognition accuracies. Hence, an improved HIR system has been proposed in this paper that combines 2D and 3D features extracted using machine learning and deep learning techniques. These discriminative features result in accurate classification and help avoid misclassification of similar interactions. Ten keyframes have been extracted from each video to reduce computational complexity. Next, these frames have been preprocessed using image normalization and noise removal techniques. The Region Of Interest (ROI), which contains the two humans involved in the interaction, has been extracted using motion detection. Then, the human silhouettes have been segmented using the GrabCut algorithm. Next, the extracted silhouettes have been converted into 3D meshes and their heat kernel signatures (HKS) have been obtained to extract key body points. A Convolutional Neural Network (CNN) has been used to extract full-body features from 2D full-body silhouettes. Then, topological and geometric features have been extracted from the key body points. Finally, the combined feature vector has been fed into Long Short-Term Memory (LSTM) and each interaction has been recognized using a Softmax classifier. The proposed system has been validated via extensive experimentation on three challenging RGB+D datasets. The recognition accuracies of 91.63%, 90.54%, and 90.13% have been achieved with the SBU Kinect Interaction, NTU RGB+D, and ISR-UoL 3D social activity datasets respectively. The results of extensive experiments performed on the proposed system suggest that it can be used effectively for various applications, such as security, surveillance, health monitoring, and assisted living.

A Hybrid Method for Human Interaction Recognition Using Spatio-temporal Interest Points

Recognizing human interactions by genetic algorithm-based random forest spatio-temporal correlation

Salient Pairwise Spatio-Temporal Interest Points for Real-Time Activity Recognition.

Recognizing human actions using a new descriptor based on spatial-temporal interest points and weighted-output classifier.

Human Action Recognition Using Multi-Velocity STIPs and Motion Energy Orientation Histogram.

An LSTM-Based Approach for Understanding Human Interactions Using Hybrid Feature Descriptors Over Depth Sensors

Human Interaction Recognition Based on Whole-Individual Detection.

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

Cross-network User Identification Based on Multiple Spatio-Temporal Trajectory Features

Matching user accounts with spatio-temporal awareness across social networks

Action Recognition Using Spatiotemporal Features and Hybrid Generative/Discriminative Models

Robust Human Action Recognition Based on Spatio-Temporal Descriptors and Motion Temporal Templates

Human Behavior Recognition Based on CNN-LSTM Hybrid and Multi-Sensing Feature Information Fusion

Two-stream Multi-level Dynamic Point Transformer for Two-person Interaction Recognition

Human Activity Recognition Based on Pose Points Selection

A Hierarchical Spatio-Temporal Model for Human Activity Recognition.

Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition

Human Interaction Recognition Method Based on Parallel Multi-Feature Fusion Network

Human Motion Recognition Based on Spatio-Temporal Interest Point

Poster: Attention-based Spatio-Temporal Model for HAR Using Multivariate Time Series.

Recognising human interaction from videos by a discriminative model