Abstract:In computer vision, human action recognition is a hot topic, popularized by the development of deep learning. Deep learning models typically accept video input without prior processing and train them to achieve recognition. However, conducting preliminary motion analysis can be beneficial in directing the model training to prioritize the motion of individuals with less priority for the environment in which the action occurs. This paper puts forth a novel methodology for human action recognition based on motion information that employs transfer-learning techniques. The proposed method comprises four stages: (1) human detection and tracking, (2) motion estimation, (3) feature extraction, and (4) action recognition using a two-stream model. In order to develop this work, a customized dataset was utilized, comprising videos of diverse actions (e.g., walking, running, cycling, drinking, and falling) extracted from multiple public sources and websites, including Pexels and MixKit. This realistic and diverse dataset allowed for a comprehensive evaluation of the proposed method, demonstrating its effectiveness in different scenarios and conditions. Furthermore, the performance of seven pre-trained models for feature extraction was evaluated. The models analyzed were Inception-v3, MobileNet-v2, MobileNet-v3-L, VGG-16, VGG-19, Xception, and ConvNeXt-L. The results demonstrated that the ConvNeXt-L model yielded the most optimal outcomes. Furthermore, using pre-trained models for feature extraction facilitated the training process on a personal computer with a single graphics processing unit, achieving an accuracy of 94.9%. The experimental findings and outcomes suggest that integrating motion information enhances action recognition performance.

Deep Learning-Based Human Action Recognition in Videos

Human Action Recognition Using Deep Learning Methods.

Human Action Recognition From Digital Videos Based on Deep Learning.

Deep Learning Methods for Human Behavior Recognition.

Deep Neural Networks in Video Human Action Recognition: A Review

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Complex Human Action Recognition Using a Hierarchical Feature Reduction and Deep Learning-Based Method

A very deep sequences learning approach for human action recognition

A New Depth Residual Network Combined Recurrent with Residual Structure for Human Action Recognition from Videos

Human action recognition using attention based LSTM network with dilated CNN features

Deep Learning-Based Recognition and Visualization of Human Motion Behavior

Action Recognition in Videos through a Transfer-Learning-Based Technique

Learning and Recognizing Human Action from Skeleton Movement with Deep Residual Neural Networks

DB-LSTM: Densely-connected Bi-directional LSTM for Human Action Recognition

Video-based Human Action Recognition using Deep Learning: A Review

Human Action Recognition Based on Hierarchical Multi-Scale Adaptive Conv-Long Short-Term Memory Network

Video Based Action Recognition Using Spatial and Temporal Feature

Stratified Pooling Based Deep Convolutional Neural Networks for Human Action Recognition

Channel Attention-Based Approach with Autoencoder Network for Human Action Recognition in Low-Resolution Frames

Action recognition with temporal scale-invariant deep learning framework

Deep Learning Based Human Action Recognition:A Survey