Abstract:This article presents a video‐based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The main contribution of the system is an innovative AI model for detection of traffic anomalies with the help of the automated camera calibration, enabling instant camera pose estimation of traffic agents onto the road/intersection surface. The complete system achieves leading anomaly detection results on public datasets and it is deployed and validated on the roads of Helmond town in The Netherlands. Understanding the behaviour of traffic participants within the geo‐spatial context of road/intersection topology is a vital prerequisite for any smart ITS application. This article presents a video‐based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The system solves the challenge of geo‐spatial mapping of captured visual data onto the road/intersection topology by semantic analysis of aerial data. Additionally, the automated camera calibration component enables instant camera pose estimation to map traffic agents onto the road/intersection surface accurately. A novel aspect is approaching the anomaly detection problem by AI analysis of both the spatio‐temporal visual clues and the geo‐spatial trajectories for all type of traffic participants, such as pedestrians, bicyclists, and vehicles. This enables recognition of anomalies related to either traffic‐rule violations, for example, jaywalking, improper turns, zig‐zag driving, unlawful stops, or behavioural anomalies: littering, accidents, falling, vandalism, violence, infrastructure collapse etc. The method achieves leading anomaly detection results on benchmark datasets World Cup 2014, UCF‐Crime, XD‐Violence, and ShanghaiTech. All the obtained results are streamed and rendered in real‐time by the developed TGX digital twin visualizer. The complete system has been deployed and validated on the roads of Helmond town in The Netherlands.

AtoMixer: Atom-based Interactive Visual Exploration of Traffic Surveillance Data.

&Lt;title>automatic Traffic Real-Time Analysis System Based on Video</title>

Visual Analysis of Traffic Data Based on Topic Modeling (chinavis 2017).

A Visual Analytics Approach for Traffic Flow Prediction Ensembles.

From Time to Space: Automatic Annotation of Unmarked Traffic Scene Based on Trajectory Data.

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

A visual reasoning approach for data-driven transport assessment on urban roads

SemanticTraj: A New Approach to Interacting with Massive Taxi Trajectories.

Visual Exploration of Sparse Traffic Trajectory Data.

Interactive and Collaborative Visual Analysis on Traffic Sensor Data.

Interactive Visual Analysis of Fake Plate Vehicles Detection

Spatiotemporal Analysis of Static and Dynamic Traffic Elements from Road Scenes.

Visual Analytics of the Spatio-temporal Multidimensional Air Monitoring Data

Geo‐spatial traffic behaviour analysis and anomaly detection for ITS applications

TripVista: Triple Perspective Visual Trajectory Analytics and Its Application on Microscopic Traffic Data at a Road Intersection

A Trip Building and Chaining Methodology Using Traffic Surveillance Data

A Visual Analytics System for Exploring, Monitoring, and Forecasting Road Traffic Congestion

A-TEAM: Advanced Traffic Event Analysis and Management Platform for Transportation Data-Driven Problem Solving

Understanding Urban Dynamics from Massive Mobile Traffic Data.

Text-Driven Traffic Anomaly Detection with Temporal High-Frequency Modeling in Driving Videos

A Method of Intersection Traffic Volume Statistics Using Aerial Video