Abstract:Automating tasks through human activity recognition in video data has become increasingly vital. Deep learning has yielded versatile activity recognition systems applicable in surveillance, healthcare analysis, sports, and human–computer interaction. Despite various proposed video-based activity recognition techniques over the years, the reliance over RGB frames, accompanied by other modalities like joint locations and depth maps, often proves less effective compared to multimodal methods. In response to this challenge, our paper introduces a competitive approach for identifying human activity in video frames. Leveraging a Convolutional Long Short-Term Memory (Conv-LSTM) network and a novel pre-processing step involving a Human Segmentation network, our method accentuates human subjects in each frame using segmentation maps. These highlighted frames undergo further processing through Convolutional Neural Networks (CNNs) to learn feature vectors, without including other modalities with RGB frames directly. The learned features are then subjected to Long Short-Term Memory (LSTM) units for comprehending sequential video data and drawing meaningful inferences. The proposed methodology undergoes rigorous testing on three publicly available datasets—KARD, MSR Daily Activity, and SBU-Interactions. Remarkably, our approach outperforms similar state-of-the-art methods, achieving benchmark accuracy scores exceeding 98% on MSR Daily Activity and 99% on KARD and SBU-Interactions datasets. In essence, our method not only provides a competitive solution for human activity recognition in video frames but also contributes to advancing the field by integrating Conv-LSTM networks and innovative pre-processing techniques. The comprehensive evaluation on multiple datasets underlines the robustness and superior performance of our proposed approach.

Recognize Human Activities From Multi-Part Missing Videos

A Method of Simultaneously Action Recognition and Video Segmentation of Video Streams.

Human Activities Prediction by Learning Combinatorial Sparse Representations.

Spatio-Temporal Triangular-Chain Crf For Activity Recognition

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

M&M: Recognizing Multiple Co-evolving Activities from Multi-source Videos

Action unit detection and key frame selection for human activity prediction.

Continuous Action Recognition and Segmentation in Untrimmed Videos

Real Time Human Action Recognition in a Long Video Sequence

A human activity recognition framework in videos using segmented human subject focus

A Pursuit of Temporal Accuracy in General Activity Detection.

Activity recognition through multi-scale motion detail analysis.

A hierarchical feature graph matching method for recognition of complex human activities

Action Recognition By Learning Deep Multi-Granular Spatio-Temporal Video Representation

Action Recognition by Hidden Temporal Models

Breaking Video into Pieces for Action Recognition

Continuous Human Action Recognition in Real Time

Action Recognition in Video Using Human Keypoint Detection

Locating and Recognizing Multiple Human Actions by Searching for Maximum Score Subsequences

A Hierarchical Video Description for Complex Activity Understanding.

Action Recognition Using Hybrid Feature Descriptor And Vlad Video Encoding