Abstract:Automating tasks through human activity recognition in video data has become increasingly vital. Deep learning has yielded versatile activity recognition systems applicable in surveillance, healthcare analysis, sports, and human–computer interaction. Despite various proposed video-based activity recognition techniques over the years, the reliance over RGB frames, accompanied by other modalities like joint locations and depth maps, often proves less effective compared to multimodal methods. In response to this challenge, our paper introduces a competitive approach for identifying human activity in video frames. Leveraging a Convolutional Long Short-Term Memory (Conv-LSTM) network and a novel pre-processing step involving a Human Segmentation network, our method accentuates human subjects in each frame using segmentation maps. These highlighted frames undergo further processing through Convolutional Neural Networks (CNNs) to learn feature vectors, without including other modalities with RGB frames directly. The learned features are then subjected to Long Short-Term Memory (LSTM) units for comprehending sequential video data and drawing meaningful inferences. The proposed methodology undergoes rigorous testing on three publicly available datasets—KARD, MSR Daily Activity, and SBU-Interactions. Remarkably, our approach outperforms similar state-of-the-art methods, achieving benchmark accuracy scores exceeding 98% on MSR Daily Activity and 99% on KARD and SBU-Interactions datasets. In essence, our method not only provides a competitive solution for human activity recognition in video frames but also contributes to advancing the field by integrating Conv-LSTM networks and innovative pre-processing techniques. The comprehensive evaluation on multiple datasets underlines the robustness and superior performance of our proposed approach.

Human activity recognition for video surveillance

Activity Recognition Using A Combination of Category Components And Local Models for Video Surveillance.

Multi-dimensional CNN Based Feature Extraction with Feature Fusion and SVM for Human Activity Recognition in Surveillance Videos

Group Event Detection for Video Surveillance

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Recognition of Human Activities Using Svm Multi-Class Classifier

A Hierarchical Spatio-Temporal Model for Human Activity Recognition.

Human activity recognition based on the combined SVM&HMM

Group Event Detection with a Varying Number of Group Members for Video Surveillance

A human activity recognition framework in videos using segmented human subject focus

Recognition of Humans and Their Activities Using Video

Human action recognition in videos using motion impression image

Learning Dynamic Spatio-Temporal Relations for Human Activity Recognition.

View-invariant Action Recognition in Surveillance Videos

Detecting Group Activities with Multi-Camera Context

Action Recognition Using Context and Appearance Distribution Features

A Multimodal Fusion Approach for Human Activity Recognition

Activity Recognition Via Feature Decomposition

Human Activity Recognition by Multi-Modal Context Fusion

View-invariant feature discovering for multi-camera human action recognition