Abstract:Monitoring continuously captured surveillance videos is a challenging and time consuming task. To assist this issue, a new framework is introduced that applies anomaly detection, semantic annotation and provides a concept-based search interface. In particular, novel optical flow based features are used for abnormal crowd behaviour detection. Then, processed surveillance videos are annotated using a new semantic metadata model based on multimedia standards using Semantic Web technologies. In this way, globally inter-operable metadata about abnormal crowd behaviours are generated. Finally, for the first time, based on crowd behaviours, a novel concept-based semantic search interface is proposed. In the proposed interface, along with search results (video segments), statistical data about crowd behaviours are also presented. With extensive user studies, it is demonstrated that the proposed concept-based semantic search interface enables efficient search and analysis of abnormal crowd behaviours. Although there are existing works to achieve (a) crowd anomaly detection, (b) semantic annotation and (c) semantic search interface, none of the existing works combine these three system components in a novel framework like the one proposed in this paper. In each system component, we introduce contributions to the field as well as use the Semantic Web technologies to combine and standardize output of different system components; output of the anomaly detection is automatically annotated with metadata and stored to a semantic database. When continuous surveillance videos are processed, only the semantic database is updated. Finally, the user interface queries the updated database for searching/analyzing surveillance videos without changing any coding. Thus, the framework supports re-usability. This paper explains and evaluates different components of the framework.

Unsupervised group-based crowd dynamic behavior detection and tracking in online video sequences

Counting moving people in crowds using motion statistics of feature-points

Public Place Crowd Transaction Monitoring System

Analysis of Unstructured High-Density Crowded Scenes for Crowd Monitoring

Automation of Pedestrian Tracking in a Crowded Situation

A Review on Crowd Behavior Analysis Methods for Video Surveillance

Evolving graph-based video crowd anomaly detection

Real-time crowd behavior recognition in surveillance videos based on deep learning methods

Online anomaly detection in crowd scenes via structure analysis

Anomaly Detection and Localisation in the Crowd Scenes Using a Block-Based Social Force Model

Abnormalities detection and tracking of crowd using kinematics approach in a video stream

Abnormal Behavior Learning Based on Edge Computing toward a Crowd Monitoring System

Fast and accurate detection and localization of abnormal behavior in crowded scenes

Energy Level-Based Abnormal Crowd Behavior Detection

Detecting and Quantifying Crowd-Level Abnormal Behaviors in Crowd Events

Abnormal Event Detection and Location for Dense Crowds using Repulsive Forces and Sparse Reconstruction

Crowd dynamics analysis and behavior recognition in surveillance videos based on deep learning

Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering

Anomaly Detection and Localization in Crowded Scenes Using Short-Term Trajectories

Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection

A novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos