Abstract:Over the past few years, automatic recognition of human interactions has drawn significant attention from researchers working in the field of Artificial Intelligence (AI). And feature extraction is one of the most critical tasks in developing efficient Human Interaction Recognition (HIR) systems. Moreover, recent researches in computer vision suggest that robust features lead to higher recognition accuracies. Hence, an improved HIR system has been proposed in this paper that combines 2D and 3D features extracted using machine learning and deep learning techniques. These discriminative features result in accurate classification and help avoid misclassification of similar interactions. Ten keyframes have been extracted from each video to reduce computational complexity. Next, these frames have been preprocessed using image normalization and noise removal techniques. The Region Of Interest (ROI), which contains the two humans involved in the interaction, has been extracted using motion detection. Then, the human silhouettes have been segmented using the GrabCut algorithm. Next, the extracted silhouettes have been converted into 3D meshes and their heat kernel signatures (HKS) have been obtained to extract key body points. A Convolutional Neural Network (CNN) has been used to extract full-body features from 2D full-body silhouettes. Then, topological and geometric features have been extracted from the key body points. Finally, the combined feature vector has been fed into Long Short-Term Memory (LSTM) and each interaction has been recognized using a Softmax classifier. The proposed system has been validated via extensive experimentation on three challenging RGB+D datasets. The recognition accuracies of 91.63%, 90.54%, and 90.13% have been achieved with the SBU Kinect Interaction, NTU RGB+D, and ISR-UoL 3D social activity datasets respectively. The results of extensive experiments performed on the proposed system suggest that it can be used effectively for various applications, such as security, surveillance, health monitoring, and assisted living.

Real-Time Human Action Recognition Using DMMs-Based LBP and EOH Features

Human Action Recognition Based on DMMs, HOGs and Contourlet Transform

DMMs-Based Multiple Features Fusion for Human Action Recognition

Human action recognition using Adaptive Hierarchical Depth Motion Maps and Gabor filter

Action Recognition from Depth Sequences Using Weighted Fusion of 2D and 3D Auto-Correlation of Gradients Features

Recognizing actions using depth motion maps-based histograms of oriented gradients

Multi-Temporal Depth Motion Maps-Based Local Binary Patterns for 3-D Human Action Recognition

Real-Time Human Action Recognition System Using Depth Map Sequences

3D Action Recognition Using Multi-Temporal Depth Motion Maps and Fisher Vector

Online Robust Action Recognition Based on a Hierarchical Model

Human Action Recognition Using Deep Learning Methods.

Action Segmentation And Recognition Based On Depth Hog And Probability Distribution Difference

CNN-based and DTW features for human activity recognition on depth maps

Deep Convolutional Neural Networks for Action Recognition Using Depth Map Sequences

Exploring 3d Human Action Recognition: From Offline To Online

Combining Multi-scale Directed Depth Motion Maps and Log-Gabor Filters for Human Action Recognition

Depth2Action: Exploring Embedded Depth for Large-Scale Action Recognition

Combining Adaptive Hierarchical Depth Motion Maps with Skeletal Joints for Human Action Recognition

Spatial-temporal Texture Features for 3D Human Activity Recognition Using Laser-Based RGB-D Videos

Human Action Recognition with Depth Cameras

An LSTM-Based Approach for Understanding Human Interactions Using Hybrid Feature Descriptors Over Depth Sensors