Abstract:Vision-based human activity recognition (HAR) has emerged as one of the essential research areas in video analytics. Over the last decade, numerous advanced deep learning algorithms have been introduced to recognize complex human actions from video streams. These deep learning algorithms have shown impressive performance for the video analytics task. However, these newly introduced methods either exclusively focus on model performance or the effectiveness of these models in terms of computational efficiency, resulting in a biased trade-off between robustness and computational efficiency in their proposed methods to deal with challenging HAR problem. To enhance both the accuracy and computational efficiency, this paper presents a computationally efficient yet generic spatial–temporal cascaded framework that exploits the deep discriminative spatial and temporal features for HAR. For efficient representation of human actions, we propose an efficient dual attentional convolutional neural network (DA-CNN) architecture that leverages a unified channel–spatial attention mechanism to extract human-centric salient features in video frames. The dual channel–spatial attention layers together with the convolutional layers learn to be more selective in the spatial receptive fields having objects within the feature maps. The extracted discriminative salient features are then forwarded to a stacked bi-directional gated recurrent unit (Bi-GRU) for long-term temporal modeling and recognition of human actions using both forward and backward pass gradient learning. Extensive experiments are conducted on three publicly available human action datasets, where the obtained results verify the effectiveness of our proposed framework (DA-CNN+Bi-GRU) over the state-of-the-art methods in terms of model accuracy and inference runtime across each dataset. Experimental results show that the DA-CNN+Bi-GRU framework attains an improvement in execution time up to 167× in terms of frames per second as compared to most of the contemporary action-recognition methods.

Harmonizing space–time dynamics for precision in human action recognition

Human Activity Recognition Based on a Modified Capsule Network

Human Action Recognition Based on Three-Stream Network with Frame Sequence Features

Human Action Recognition Using Deep Learning Methods.

Modeling transformer architecture with attention layer for human activity recognition

A multidimensional feature fusion network based on MGSE and TAAC for video-based human action recognition

Human Action Recognition of Spatiotemporal Parameters for Skeleton Sequences Using MTLN Feature Learning Framework

Hand-crafted and deep convolutional neural network features fusion and selection strategy: An application to intelligent human action recognition

Human action recognition in immersive virtual reality based on multi‐scale spatio‐temporal attention network

A resource conscious human action recognition framework using 26-layered deep convolutional neural network

Spatiotemporal Multi-Task Network for Human Activity Understanding.

RNNs, CNNs and Transformers in Human Action Recognition: A Survey and a Hybrid Model

Human Activity Recognition Using Cascaded Dual Attention CNN and Bi-Directional GRU Framework

A Cross-Modal Learning Approach for Recognizing Human Actions

Multi-View Region Adaptive Multi-temporal DMM and RGB Action Recognition

Human action recognition using attention based LSTM network with dilated CNN features

Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation

SpikMamba: When SNN meets Mamba in Event-based Human Action Recognition

Action recognition using attention-based spatio-temporal VLAD networks and adaptive video sequences optimization

Human-centric multimodal fusion network for robust action recognition

Towards Improved Human Action Recognition Using Convolutional Neural Networks and Multimodal Fusion of Depth and Inertial Sensor Data