Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors

Nicolae-Catalin Ristea,Florinel-Alin Croitoru,Radu Tudor Ionescu,Marius Popescu,Fahad Shahbaz Khan,Mubarak Shah

2024-03-10

Abstract:We propose an efficient abnormal event detection model based on a lightweight masked auto-encoder (AE) applied at the video frame level. The novelty of the proposed model is threefold. First, we introduce an approach to weight tokens based on motion gradients, thus shifting the focus from the static background scene to the foreground objects. Second, we integrate a teacher decoder and a student decoder into our architecture, leveraging the discrepancy between the outputs given by the two decoders to improve anomaly detection. Third, we generate synthetic abnormal events to augment the training videos, and task the masked AE model to jointly reconstruct the original frames (without anomalies) and the corresponding pixel-level anomaly maps. Our design leads to an efficient and effective model, as demonstrated by the extensive experiments carried out on four benchmarks: Avenue, ShanghaiTech, UBnormal and UCSD Ped2. The empirical results show that our model achieves an excellent trade-off between speed and accuracy, obtaining competitive AUC scores, while processing 1655 FPS. Hence, our model is between 8 and 70 times faster than competing methods. We also conduct an ablation study to justify our design. Our code is freely available at: <a class="link-external link-https" href="https://github.com/ristea/aed-mae" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve several key problems in video abnormal event detection:

1. **Data scarcity problem**: Since abnormal events rarely occur in the real world, it is very difficult to collect representative abnormal event datasets for training deep - learning models. This makes it impossible to train abnormal detectors in a fully - supervised setting. To this end, the paper proposes a method of increasing the amount of training data by superimposing synthetic abnormal events on normal training videos, thereby achieving open - set supervised learning.

2. **Problem of excessive model generalization ability**: Traditional auto - encoders (AEs) exhibit overly high reconstruction accuracy when dealing with abnormal data, which is contrary to the requirements of abnormal detection tasks. The paper enhances the model's ability to recognize abnormal events by introducing a weighting mechanism to emphasize the importance of tokens with high motion gradients and by using self - distillation techniques to utilize the output differences between the teacher model and the student model.

3. **Improving model efficiency and accuracy**: In order to significantly improve the processing speed while maintaining high accuracy, the paper designs a lightweight masked auto - encoder model that can achieve a processing speed of 1,655 frames per second on four benchmark datasets (Avenue, ShanghaiTech, UBnormal, and UCSD Ped2), while achieving AUC scores comparable to or better than existing methods. In addition, by using a shared encoder for self - distillation, the computational overhead of the model is further reduced.

In summary, this paper mainly solves the problems of data scarcity, excessive model generalization ability, and how to improve model accuracy while maintaining high efficiency in video abnormal event detection.