Abstract:Individual abnormal behaviors vary depending on crowd sizes, contexts, and scenes. Challenges such as partial occlusions, blurring, a large number of abnormal behaviors, and camera viewing occur in large-scale crowds when detecting, tracking, and recognizing individuals with abnormalities. In this paper, our contribution is two-fold. First, we introduce an annotated and labeled large-scale crowd abnormal behavior Hajj dataset, HAJJv2. Second, we propose two methods of hybrid convolutional neural networks (CNNs) and random forests (RFs) to detect and recognize spatio-temporal abnormal behaviors in small and large-scale crowd videos. In small-scale crowd videos, a ResNet-50 pre-trained CNN model is fine-tuned to verify whether every frame is normal or abnormal in the spatial domain. If anomalous behaviors are observed, a motion-based individual detection method based on the magnitudes and orientations of Horn–Schunck optical flow is proposed to locate and track individuals with abnormal behaviors. A Kalman filter is employed in large-scale crowd videos to predict and track the detected individuals in the subsequent frames. Then, means and variances as statistical features are computed and fed to the RF classifier to classify individuals with abnormal behaviors in the temporal domain. In large-scale crowds, we fine-tune the ResNet-50 model using a YOLOv2 object detection technique to detect individuals with abnormal behaviors in the spatial domain. The proposed method achieves 99.76% and 93.71% of average area under the curves (AUCs) on two public benchmark small-scale crowd datasets, UMN and UCSD, respectively, while the large-scale crowd method achieves 76.08% average AUC using the HAJJv2 dataset. Our method outperforms state-of-the-art methods using the small-scale crowd datasets with a margin of 1.66%, 6.06%, and 2.85% on UMN, UCSD Ped1, and UCSD Ped2, respectively. It also produces an acceptable result in large-scale crowds.

Optimized deep maxout for crowd anomaly detection: A hybrid optimization-based model

Crowd Anomaly Detection in Video Frames Using Fine-Tuned AlexNet Model

Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes

Human crowd behaviour analysis based on video segmentation and classification using expectation–maximization with deep learning architectures

Detecting and Quantifying Crowd-Level Abnormal Behaviors in Crowd Events

An Adaptive Training-less System for Anomaly Detection in Crowd Scenes

Video anomaly detection based on scene classification

Anomaly Detection and Localisation in the Crowd Scenes Using a Block-Based Social Force Model

Plug-and-Play Anomaly Detection with Expectation Maximization Filtering

Crowd Characterization in Surveillance Videos Using Deep-Graph Convolutional Neural Network

Evolving graph-based video crowd anomaly detection

Boosting Detection in Crowd Analysis via Underutilized Output Features

Leveraging YOLOv5s with optimization‐based effective anomaly detection in pedestrian walkways

Influence-aware Attention Networks for Anomaly Detection in Surveillance Videos

Human activity-based anomaly detection and recognition by surveillance video using kernel local component analysis with classification by deep learning techniques

Visual data mining for crowd anomaly detection using artificial bacteria colony

Object-Guided and Motion-Refined Attention Network for Video Anomaly Detection

Crowd Saliency Detection via Global Similarity Structure

Hybrid Classifiers for Spatio-Temporal Abnormal Behavior Detection, Tracking, and Recognition in Massive Hajj Crowds

Postures anomaly tracking and prediction learning model over crowd data analytics

Video abnormal event detection based on fast sparse coding and surprise computation