Abstract:Vision-based human activity recognition (HAR) has emerged as one of the essential research areas in video analytics. Over the last decade, numerous advanced deep learning algorithms have been introduced to recognize complex human actions from video streams. These deep learning algorithms have shown impressive performance for the video analytics task. However, these newly introduced methods either exclusively focus on model performance or the effectiveness of these models in terms of computational efficiency, resulting in a biased trade-off between robustness and computational efficiency in their proposed methods to deal with challenging HAR problem. To enhance both the accuracy and computational efficiency, this paper presents a computationally efficient yet generic spatial–temporal cascaded framework that exploits the deep discriminative spatial and temporal features for HAR. For efficient representation of human actions, we propose an efficient dual attentional convolutional neural network (DA-CNN) architecture that leverages a unified channel–spatial attention mechanism to extract human-centric salient features in video frames. The dual channel–spatial attention layers together with the convolutional layers learn to be more selective in the spatial receptive fields having objects within the feature maps. The extracted discriminative salient features are then forwarded to a stacked bi-directional gated recurrent unit (Bi-GRU) for long-term temporal modeling and recognition of human actions using both forward and backward pass gradient learning. Extensive experiments are conducted on three publicly available human action datasets, where the obtained results verify the effectiveness of our proposed framework (DA-CNN+Bi-GRU) over the state-of-the-art methods in terms of model accuracy and inference runtime across each dataset. Experimental results show that the DA-CNN+Bi-GRU framework attains an improvement in execution time up to 167× in terms of frames per second as compared to most of the contemporary action-recognition methods.

Hand-crafted and deep convolutional neural network features fusion and selection strategy: An application to intelligent human action recognition

A resource conscious human action recognition framework using 26-layered deep convolutional neural network

Feature Fusion for Human Activity Recognition using Parameter-Optimized Multi-Stage Graph Convolutional Network and Transformer Models

Towards Improved Human Action Recognition Using Convolutional Neural Networks and Multimodal Fusion of Depth and Inertial Sensor Data

Human action recognition using attention based LSTM network with dilated CNN features

Multi-dimensional CNN Based Feature Extraction with Feature Fusion and SVM for Human Activity Recognition in Surveillance Videos

Inertial Sensor Data To Image Encoding For Human Action Recognition

A deep multimodal network based on bottleneck layer features fusion for action recognition

Human Action Recognition Using Deep Multilevel Multimodal (M2) Fusion of Depth and Inertial Sensors

Vision Transformer and Deep Sequence Learning for Human Activity Recognition in Surveillance Videos

Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation

Exploring Hybrid Spatio-Temporal Convolutional Networks for Human Action Recognition.

A Novel Multi-Stage Training Approach for Human Activity Recognition From Multimodal Wearable Sensor Data Using Deep Neural Network

Local-aware spatio-temporal attention network with multi-stage feature fusion for human action recognition

Using a Selective Ensemble Support Vector Machine to Fuse Multimodal Features for Human Action Recognition

Human Activity Recognition Using Attention-Mechanism-Based Deep Learning Feature Combination

Human Activity Recognition Using Cascaded Dual Attention CNN and Bi-Directional GRU Framework

A multidimensional feature fusion network based on MGSE and TAAC for video-based human action recognition

Stratified Pooling Based Deep Convolutional Neural Networks for Human Action Recognition

Integrating Features for Recognizing Human Activities through Optimized Parameters in Graph Convolutional Networks and Transformer Architectures