Abstract:Violent behavior detection (VioBD), as a special action recognition task, aims to detect violent behaviors in videos, such as mutual fighting and assault. Some progress has been made in the research of violence detection, but the existing methods have poor real-time performance and the algorithm performance is limited by the interference of complex backgrounds and the occlusion of dense crowds. To solve the above problems, we propose an end-to-end real-time violence detection framework based on 2D CNNs. First, we propose a lightweight skeletal image (SI) as the input modality, which can obtain the human body posture information and richer contextual information, and at the same time remove the background interference. As tested, at the same accuracy, the resolution of SI modality is only one-third of that of RGB modality, which greatly improves the real-time performance of model training and inference, and at the same resolution, SI modality has higher inaccuracy. Second, we also design a parallel prediction module (PPM), which can simultaneously obtain the single image detection results and the inter-frame motion information of the video, which can improve the real-time performance of the algorithm compared with the traditional "detect the image first, understand the video later" mode. In addition, we propose an auxiliary parameter generation module (APGM) with both efficiency and accuracy, APGM is a 2D CNNs-based video understanding module for weighting the spatial information of the video features, processing speed can reach 30–40 frames per second, and compared with models such as CNN-LSTM (Iqrar et al., Aamir: Cnn-lstm based smart real-time video surveillance system. In: 2022 14th International Conference on Mathematics, Actuarial, Science, Computer Science and Statistics (MACS), pages 1–5. IEEE, 2022) and Ludl et al. (Cristóbal: Simple yet efficient real-time pose-based action recognition. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pages 581–588. IEEE, 1999), the propagation effect speed can be increased by an average of frames per second per group of clips, which further improves the video motion detection efficiency and accuracy, greatly improving real-time performance. We conducted experiments on some challenging benchmarks, and RVBDN can maintain excellent speed and accuracy in long-term interactions, and are able to meet real-time requirements in methods for violence detection and spatio-temporal action detection. Finally, we update our proposed new dataset on violence detection images (violence image dataset). Dataset is available at https://github.com/ChinaZhangPeng/Violence-Image-Dataset

Violence behavior recognition of two-cascade temporal shift module with attention mechanism

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

Campus Abnormal Behavior Recognition With Temporal Segment Transformers

Human Behavior Recognition Based on Attention Mechanism and Bottleneck Residual Dual-Path Spatiotemporal Graph Convolutional Network

Violent Video Recognition by Using Sequential Image Collage

A spatio-temporal model for violence detection based on spatial and temporal attention modules and 2D CNNs

Attention-Based Deep Spiking Neural Networks for Temporal Credit Assignment Problems.

Research into the Applications of a Multi-Scale Feature Fusion Model in the Recognition of Abnormal Human Behavior

Efficiently adapting large pre-trained models for real-time violence recognition in smart city surveillance

Towards Real-world Violence Recognition via Efficient Deep Features and Sequential Patterns Analysis

A study on attention-based LSTM for abnormal behavior recognition with variable pooling

Efficient Aggressive Behavior Recognition of Pigs Based on Temporal Shift Module

An end-to-end framework for real-time violent behavior detection based on 2D CNNs

A Violence Detection Approach Based On Spatio-Temporal Hypergraph Transition

Utilizing Deep Learning Models to Develop a Human Behavior Recognition System for Vision-Based School Violence Detection

Efficient Human Violence Recognition for Surveillance in Real Time

A Frame-Based Feature Model for Violence Detection from Surveillance Cameras Using ConvLSTM Network

Research Progress on the Human Behavior Recognition Based on Machine Learning Methods

Mobile Neural Architecture Search Network and Convolutional Long Short-Term Memory-Based Deep Features Toward Detecting Violence from Video

Research on TCN Model Based on SSARF Feature Selection in the Field of Human Behavior Recognition

Research on Character Behavior Recognition Based on Local Spatio-temporal Relationship in Surveillance Video