Abstract:Human abnormal behavior detection is important to ensure public safety and prevent unwanted incidents. Currently, recognition systems for human abnormal behavior adopt neural network models and perform standard 1-of-N majority voting procedures. However, recognizing human abnormal behaviors can be challenging due to lengthy and numerous video datasets and the limitations of existing methods that rely on predefined categories and scenarios. This study proposed a novel method named Visual Text Contrastive Learning (VTCL) for identifying abnormal human behavior in campus settings. The proposed model emphasizes semantic information from automatically labeled properties text and videos of abnormal behaviors, moving beyond simple numerical representations. The proposed method integrates the cross and multi-frame methods within the visual branch to improve spatial and temporal performance. In the textual branch, the proposed prompting technique captures the contextual backdrop of abnormal behaviors to enrich supervision with behavioral semantic information. Then, the model learns the visual-text features to enhance the learning process through contrastive learning techniques. In addition, this work also presented a new study to explore zero-shot campus abnormal behavior recognition (CABR). It lays the foundation for unlocking the implementation of highly available and robust CABR for multiple and even new scenarios. The proposed VTCL model demonstrated a Top-1 accuracy of 86.92% and a Top-5 accuracy of 98.14% on the CABR50 dataset, including fifty abnormal behaviors on campus, with competitive computational complexity. Furthermore, the zero-shot performance of the proposed model showed competitive outcomes when evaluated on additional datasets, including CABRZ6 and UCF-101.

A weakly supervised framework for abnormal behavior detection and localization in crowded scenes.

Abnormal Behavior Detection Based on Regional Optical Flow

Sparse representation for robust abnormality detection in crowded scenes

Context-aware Local Abnormality Detection in Crowded Scene

Anomaly Detection and Localization in Crowded Scenes by Motion-field Shape Description and Similarity-based Statistical Learning

Abnormal event detection and localization in crowded scenes based on PCANet

Crowd-level Abnormal Behavior Detection via Multi-scale Motion Consistency Learning

Abnormal Event Detection Based on Multi-scale Markov Random Field.

Abnormal behavior recognition using 3D-CNN combined with LSTM

Abnormal crowd behavior detection using high-frequency and spatio-temporal features

An Abnormal Behavior Detection Method of Video Crowds and Vehicles Based on Deep Learning

Abnormal Event Detection and Location for Dense Crowds using Repulsive Forces and Sparse Reconstruction

Online anomaly detection in crowd scenes via structure analysis

Real-time and Accurate Abnormal Behavior Detection in Videos

Spatial-temporal Convolutional Neural Networks for Anomaly Detection and Localization in Crowded Scenes

Unsupervised fast anomaly detection in crowds.

Laser-based Intelligent Surveillance and Abnormality Detection in Extremely Crowded Scenarios

Anomaly Detection In Crowded Scenes By Sl-Hof Descriptor And Foreground Classification

Anomaly Detection through Spatio-temporal Context Modeling in Crowded Scenes

Weakly-supervised Anomaly Detection with a Sub-Max Strategy.

Novel multimodal contrast learning framework using zero-shot prediction for abnormal behavior recognition