Abstract:This article presents a video‐based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The main contribution of the system is an innovative AI model for detection of traffic anomalies with the help of the automated camera calibration, enabling instant camera pose estimation of traffic agents onto the road/intersection surface. The complete system achieves leading anomaly detection results on public datasets and it is deployed and validated on the roads of Helmond town in The Netherlands. Understanding the behaviour of traffic participants within the geo‐spatial context of road/intersection topology is a vital prerequisite for any smart ITS application. This article presents a video‐based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The system solves the challenge of geo‐spatial mapping of captured visual data onto the road/intersection topology by semantic analysis of aerial data. Additionally, the automated camera calibration component enables instant camera pose estimation to map traffic agents onto the road/intersection surface accurately. A novel aspect is approaching the anomaly detection problem by AI analysis of both the spatio‐temporal visual clues and the geo‐spatial trajectories for all type of traffic participants, such as pedestrians, bicyclists, and vehicles. This enables recognition of anomalies related to either traffic‐rule violations, for example, jaywalking, improper turns, zig‐zag driving, unlawful stops, or behavioural anomalies: littering, accidents, falling, vandalism, violence, infrastructure collapse etc. The method achieves leading anomaly detection results on benchmark datasets World Cup 2014, UCF‐Crime, XD‐Violence, and ShanghaiTech. All the obtained results are streamed and rendered in real‐time by the developed TGX digital twin visualizer. The complete system has been deployed and validated on the roads of Helmond town in The Netherlands.

Anomaly Candidate Identification and Starting Time Estimation of Vehicles from Traffic Videos.

Anomaly Detection in Traffic Surveillance Videos with GAN-based Future Frame Prediction

Dual-Modality Vehicle Anomaly Detection via Bilateral Trajectory Tracing

Challenges in Time-Stamp Aware Anomaly Detection in Traffic Videos

Good Practices and A Strong Baseline for Traffic Anomaly Detection

Anomaly Detection in Traffic Scenes via Spatial-aware Motion Reconstruction

Anomaly Detection and Localization in Crowded Scenes Using Short-Term Trajectories

Text-Driven Traffic Anomaly Detection with Temporal High-Frequency Modeling in Driving Videos

Detection of Anomalous Traffic Patterns and Insight Analysis from Bus Trajectory Data

DoTA: Unsupervised Detection of Traffic Anomaly in Driving Videos

Learning Representation for Anomaly Detection of Vehicle Trajectories

Traffic Anomaly Detection in Intelligent Transport Applications with Time Series Data Using Informer

ANOMALY DETECTION AND LOCALIZATION: A NOVEL TWO-PHASE FRAMEWORK BASED ON TRAJECTORY-LEVEL CHARACTERISTICS

AnomalyNet: a spatiotemporal motion-aware CNN approach for detecting anomalies in real-world autonomous surveillance

A Framework of Abnormal Behavior Detection and Classification Based on Big Trajectory Data for Mobile Networks

Deep learning detection of anomalous patterns from bus trajectories for traffic insight analysis

Anomalous Motion Detection on Highway Using Deep Learning

uTRAND: Unsupervised Anomaly Detection in Traffic Trajectories

Anomaly Detection and Localization in Crowded Scenes by Motion-field Shape Description and Similarity-based Statistical Learning

Geo‐spatial traffic behaviour analysis and anomaly detection for ITS applications

Anomaly Detection in Connected and Autonomous Vehicle Trajectories Using LSTM Autoencoder and Gaussian Mixture Model