Abstract:Over the past few years, automatic recognition of human interactions has drawn significant attention from researchers working in the field of Artificial Intelligence (AI). And feature extraction is one of the most critical tasks in developing efficient Human Interaction Recognition (HIR) systems. Moreover, recent researches in computer vision suggest that robust features lead to higher recognition accuracies. Hence, an improved HIR system has been proposed in this paper that combines 2D and 3D features extracted using machine learning and deep learning techniques. These discriminative features result in accurate classification and help avoid misclassification of similar interactions. Ten keyframes have been extracted from each video to reduce computational complexity. Next, these frames have been preprocessed using image normalization and noise removal techniques. The Region Of Interest (ROI), which contains the two humans involved in the interaction, has been extracted using motion detection. Then, the human silhouettes have been segmented using the GrabCut algorithm. Next, the extracted silhouettes have been converted into 3D meshes and their heat kernel signatures (HKS) have been obtained to extract key body points. A Convolutional Neural Network (CNN) has been used to extract full-body features from 2D full-body silhouettes. Then, topological and geometric features have been extracted from the key body points. Finally, the combined feature vector has been fed into Long Short-Term Memory (LSTM) and each interaction has been recognized using a Softmax classifier. The proposed system has been validated via extensive experimentation on three challenging RGB+D datasets. The recognition accuracies of 91.63%, 90.54%, and 90.13% have been achieved with the SBU Kinect Interaction, NTU RGB+D, and ISR-UoL 3D social activity datasets respectively. The results of extensive experiments performed on the proposed system suggest that it can be used effectively for various applications, such as security, surveillance, health monitoring, and assisted living.

A human activity recognition framework in videos using segmented human subject focus

Human activity recognition using deep learning approaches and single frame cnn and convolutional lstm

Residual deep gated recurrent unit-based attention framework for human activity recognition by exploiting dilated features

Human Activity Recognition Based On Video Summarization And Deep Convolutional Neural Network

Human Action Recognition From Digital Videos Based on Deep Learning.

A hybrid deep learning framework for daily living human activity recognition with cluster-based video summarization

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

A deep multimodal network based on bottleneck layer features fusion for action recognition

Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation

Video-Based Human Activity Recognition Using Deep Learning Approaches

A Multimodal Fusion Approach for Human Activity Recognition

Efficient Activity Recognition Using Lightweight CNN and DS-GRU Network for Surveillance Applications

A recent survey for human activity recoginition based on deep learning approach

Human activity recognition in RGB-D videos by dynamic images

Human action recognition using attention based LSTM network with dilated CNN features

An LSTM-Based Approach for Understanding Human Interactions Using Hybrid Feature Descriptors Over Depth Sensors

Human Activity Recognition Based on Deep-Temporal Learning Using Convolution Neural Networks Features and Bidirectional Gated Recurrent Unit With Features Selection

Multi-view Multi-modal Approach Based on 5S-CNN and BiLSTM Using Skeleton, Depth and RGB Data for Human Activity Recognition

3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks

AI-driven behavior biometrics framework for robust human activity recognition in surveillance systems

Deep Dilation on Multimodality Time Series for Human Activity Recognition.