Abstract:Weakly supervised video anomaly detection (WS-VAD) is a crucial area in computer vision for developing intelligent surveillance systems. This system uses three feature streams: RGB video, optical flow, and audio signals, where each stream extracts complementary spatial and temporal features using an enhanced attention module to improve detection accuracy and robustness. In the first stream, we employed an attention-based, multi-stage feature enhancement approach to improve spatial and temporal features from the RGB video where the first stage consists of a ViT-based CLIP module, with top-k features concatenated in parallel with I3D and Temporal Contextual Aggregation (TCA) based rich spatiotemporal features. The second stage effectively captures temporal dependencies using the Uncertainty-Regulated Dual Memory Units (UR-DMU) model, which learns representations of normal and abnormal data simultaneously, and the third stage is employed to select the most relevant spatiotemporal features. The second stream extracted enhanced attention-based spatiotemporal features from the flow data modality-based feature by taking advantage of the integration of the deep learning and attention module. The audio stream captures auditory cues using an attention module integrated with the VGGish model, aiming to detect anomalies based on sound patterns. These streams enrich the model by incorporating motion and audio signals often indicative of abnormal events undetectable through visual analysis alone. The concatenation of the multimodal fusion leverages the strengths of each modality, resulting in a comprehensive feature set that significantly improves anomaly detection accuracy and robustness across three datasets. The extensive experiment and high performance with the three benchmark datasets proved the effectiveness of the proposed system over the existing state-of-the-art system.

FE-VAD: High-Low Frequency Enhanced Weakly Supervised Video Anomaly Detection

Video Anomaly Detection Based on Global–Local Convolutional Autoencoder

Memory-Augmented Spatial-Temporal Consistency Network for Video Anomaly Detection.

Configurable Spatial-Temporal Hierarchical Analysis for Flexible Video Anomaly Detection

Collaborative Normality Learning Framework for Weakly Supervised Video Anomaly Detection

Multi-scale Spatial-temporal Interaction Network for Video Anomaly Detection

Denoising Diffusion-Augmented Hybrid Video Anomaly Detection Via Reconstructing Noised Frames

Learning Prompt-Enhanced Context Features for Weakly-Supervised Video Anomaly Detection

Unsupervised anomaly video detection via a double-flow ConvLSTM variational autoencoder

Open-Vocabulary Video Anomaly Detection

Safeguarding Sustainable Cities: Unsupervised Video Anomaly Detection Through Diffusion-based Latent Pattern Learning

Interleaving One-Class and Weakly-Supervised Models with Adaptive Thresholding for Unsupervised Video Anomaly Detection

Multimodal Attention-Enhanced Feature Fusion-based Weekly Supervised Anomaly Violence Detection

Rethinking Prediction-Based Video Anomaly Detection from Local-Global Normality Perspective

Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach

Towards Open Set Video Anomaly Detection

TFAE: temporal feature adjustable enhancement for video anomaly detection

Multimodal and multiscale feature fusion for weakly supervised video anomaly detection

Normality learning reinforcement for anomaly detection in surveillance videos

CNN-ViT Supported Weakly-Supervised Video Segment Level Anomaly Detection

Patch Spatio-Temporal Relation Prediction for Video Anomaly Detection