Abstract:The purpose of this paper is to present an unsupervised video anomaly detection method using Optical Flow decomposition and Spatio-Temporal feature learning (OFST). This method employs a combination of optical flow reconstruction and video frame prediction to achieve satisfactory results. The proposed OFST framework is composed of two modules: the Multi-Granularity Memory-augmented Autoencoder with Optical Flow Decomposition (MG-MemAE-OFD) and a Two-Stream Network based on Spatio-Temporal feature learning (TSN-ST). The MG-MemAE-OFD module is composed of three functional blocks: optical flow decomposition, autoencoder, and multi-granularity memory networks. The optical flow decomposition block is used to extract the main motion information of objects in optical flow, and the granularity memory network is utilized to memorize normal patterns and improve the quality of the reconstructions. To predict video frames, we introduce a two-stream network based on spatiotemporal feature learning (TSN-ST), which adopts parallel standard Transformer blocks and a temporal block to learn spatiotemporal features from video frames and optical flows. The OFST combines these two modules so that the prediction error of abnormal samples is further increased due to the larger reconstruction error. In contrast, the normal samples obtain a lower reconstruction error and prediction error. Therefore, the anomaly detection capability of the method is greatly enhanced. Our proposed model was evaluated on public datasets. Specifically, in terms of the area under the curve (AUC), our model achieved an accuracy of 85.74% on the Ped1 dataset, 99.62% on the Ped2 dataset, 93.89% on the Avenue dataset, and 76.0% on the ShanghaiTech Dataset. Our experimental results show an average improvement of 1.2% compared to the current state-of-the-art.

Video Anomaly Detection Via Successive Image Frame Prediction Leveraging Optical Flows

Video Anomaly Detection By The Duality Of Normality-Granted Optical Flow

Future Frame Prediction for Anomaly Detection -- A New Baseline

An unsupervised video anomaly detection method via Optical Flow decomposition and Spatio-Temporal feature learning

A Novel Unsupervised Video Anomaly Detection Framework Based on Optical Flow Reconstruction and Erased Frame Prediction

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

Robust Unsupervised Video Anomaly Detection by Multipath Frame Prediction

Integrated Multiscale Appearance Features and Motion Information Prediction Network for Anomaly Detection

Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction

Abnormal Behavior Detection Based on Regional Optical Flow

Learning Appearance-Motion Synergy Via Memory-Guided Event Prediction for Video Anomaly Detection

Learning Appearance-motion Normality for Video Anomaly Detection.

Spatial-Temporal Graph Convolutional Network Boosted Flow-Frame Prediction for Video Anomaly Detection

Adaptive Hierarchical Motion-Focused Model for Video Prediction.

Spatio-Temporal-based Context Fusion for Video Anomaly Detection

Decoupled appearance and motion learning for efficient anomaly detection in surveillance video

Appearance-Motion united Auto-Encoder Framework for Video Anomaly Detection

Dual contrast discriminator with sharing attention for video anomaly detection

Pose-Motion Video Anomaly Detection via Memory-Augmented Reconstruction and Conditional Variational Prediction

Video Anomaly Detection and Localization Based on an Adaptive Intra-Frame Classification Network