Abstract:The integration of artificial intelligence (AI) into human activity recognition (HAR) in smart surveillance systems has the potential to revolutionize behavior monitoring. These systems analyze an individual’s physiological or behavioral features to continuously monitor and identify any unusual or suspicious activity in video streams, thereby improving security and surveillance measures. Traditional surveillance systems often rely on manual human monitoring, which is resource-intensive, error-prone, and time-consuming. To address these limitations, computer vision-based behavior biometrics has emerged as a solution for secure video surveillance applications. However, implementing automated HAR in real-world scenarios is challenging due to the diversity of human behavior, complex spatiotemporal patterns, varying viewpoints, and cluttered backgrounds. To tackle these challenges, an AI-based behavior biometrics framework is introduced that is based on a dynamic attention fusion unit (DAFU) followed by a temporal-spatial fusion (TSF) network to effectively recognize human activity in surveillance systems. In the first phase of the proposed framework, a lightweight EfficientNetB0 backbone is enhanced by the DAFU to extract human-centric salient features using a unified channel-spatial attention mechanism. In the second phase, the DAFU features with fixed sequence lengths are passed to the proposed TSF network to capture the temporal, spatial, and behavioral dependencies in video data streams. The integration of Echo-ConvLSTM in the TSF further enhances accuracy and robustness by combining temporal dependencies from the echo state network with spatial and temporal dependencies from the convolutional long short-term memory (ConvLSTM). The proposed AI-based behavior biometrics framework is evaluated using four publicly available HAR datasets (UCF101, HMDB51, UCF50, and YouTube Action), yielding higher accuracies of 98.734%, 80.342%, 98.987%, and 98.927%, which demonstrate the superior performance when compared with state-of-the-art (SOTA) methods.

CHA-Sens: an End-to-End Comprehensive Residual Convolution Framework for CSI-based Human Activity Sensing.

Innovative Dual-Decoupling CNN with Layer-wise Temporal-Spatial Attention for Sensor-Based Human Activity Recognition

Channel Attention for Sensor-based Activity Recognition: Embedding Features into All Frequencies in DCT Domain

An Efficient Diverse-Branch Convolution Scheme for Sensor-Based Human Activity Recognition

A Multi-dimensional Parallel Convolutional Connected Network Based on Multi-source and Multi-modal Sensor Data for Human Activity Recognition

Deep Neural Networks for Sensor-Based Human Activity Recognition Using Selective Kernel Convolution

Multihead-Res-SE Residual Network with Attention for Human Activity Recognition

Simultaneous Implementation Features Extraction and Recognition Using C3D Network for WiFi-based Human Activity Recognition

Environment-Robust Device-Free Human Activity Recognition With Channel-State-Information Enhancement and One-Shot Learning

A Multidimensional Parallel Convolutional Connected Network Based on Multisource and Multimodal Sensor Data for Human Activity Recognition

Dual-Stream Contrastive Learning for Channel State Information Based Human Activity Recognition

The Convolutional Neural Networks Training With Channel-Selectivity for Human Activity Recognition Based on Sensors

Large Receptive Field Attention: An Innovation in Decomposing Large-Kernel Convolution for Sensor-Based Activity Recognition

Hybrid convolution neural network with channel attention mechanism for sensor-based human activity recognition

Multiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors

Beyond Isolated Frames: Enhancing Sensor-Based Human Activity Recognition through Intra- and Inter-Frame Attention

A CSI-Based Human Activity Recognition Using Deep Learning

Human activity recognition using wearable sensors by heterogeneous convolutional neural networks

A Novel Deep Multifeature Extraction Framework Based on Attention Mechanism Using Wearable Sensor Data for Human Activity Recognition

A spatiotemporal multi-feature extraction framework with space and channel based squeeze-and-excitation blocks for human activity recognition

AI-driven behavior biometrics framework for robust human activity recognition in surveillance systems