Abstract:Individual abnormal behaviors vary depending on crowd sizes, contexts, and scenes. Challenges such as partial occlusions, blurring, a large number of abnormal behaviors, and camera viewing occur in large-scale crowds when detecting, tracking, and recognizing individuals with abnormalities. In this paper, our contribution is two-fold. First, we introduce an annotated and labeled large-scale crowd abnormal behavior Hajj dataset, HAJJv2. Second, we propose two methods of hybrid convolutional neural networks (CNNs) and random forests (RFs) to detect and recognize spatio-temporal abnormal behaviors in small and large-scale crowd videos. In small-scale crowd videos, a ResNet-50 pre-trained CNN model is fine-tuned to verify whether every frame is normal or abnormal in the spatial domain. If anomalous behaviors are observed, a motion-based individual detection method based on the magnitudes and orientations of Horn–Schunck optical flow is proposed to locate and track individuals with abnormal behaviors. A Kalman filter is employed in large-scale crowd videos to predict and track the detected individuals in the subsequent frames. Then, means and variances as statistical features are computed and fed to the RF classifier to classify individuals with abnormal behaviors in the temporal domain. In large-scale crowds, we fine-tune the ResNet-50 model using a YOLOv2 object detection technique to detect individuals with abnormal behaviors in the spatial domain. The proposed method achieves 99.76% and 93.71% of average area under the curves (AUCs) on two public benchmark small-scale crowd datasets, UMN and UCSD, respectively, while the large-scale crowd method achieves 76.08% average AUC using the HAJJv2 dataset. Our method outperforms state-of-the-art methods using the small-scale crowd datasets with a margin of 1.66%, 6.06%, and 2.85% on UMN, UCSD Ped1, and UCSD Ped2, respectively. It also produces an acceptable result in large-scale crowds.

Crowd density estimation using deep learning for Hajj pilgrimage video analytics

Scale Pyramid Network For Crowd Counting

Indoor and Outdoor Crowd Density Level Estimation with Video Analysis through Machine Learning Models

Open Dataset for Predicting Pilgrim Activities for Crowd Management During Hajj Using Wearable Sensors

Hybrid Classifiers for Spatio-Temporal Abnormal Behavior Detection, Tracking, and Recognition in Massive Hajj Crowds

Analyzing Crowd Behavior in Highly Dense Crowd Videos Using 3D ConvNet and Multi-SVM

Crowd Density Estimation Method Using Deep Learning for Passenger Flow Detection System in Exhibition Center

Hybrid Classifiers for Spatio-temporal Real-time Abnormal Behaviors Detection, Tracking, and Recognition in Massive Hajj Crowds

Generative adversarial network based abnormal behavior detection in massive crowd videos: a Hajj case study

Estimation of Crowd Density from UAV Images based on Deep Learning

LCDnet: a lightweight crowd density estimation model for real-time video surveillance

A Hajj And Umrah Location Classification System For Video Crowded Scenes

Slime Mold Optimization with Hybrid Deep Learning Enabled Crowd-Counting Approach in Video Surveillance

Human crowd behaviour analysis based on video segmentation and classification using expectation–maximization with deep learning architectures

Crowd dynamics analysis and behavior recognition in surveillance videos based on deep learning

Deep Learning Approaches for Crowd Density Estimation: A Review

Crowd counting method based on the self-attention residual network

AI-based Pilgrim Detection using Convolutional Neural Networks

Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds

Crowd density estimation via a multichannel dense grouping network

Artificial Intelligence Aided Crowd Analytics in Rail Transit Station