Abstract:This article presents a video‐based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The main contribution of the system is an innovative AI model for detection of traffic anomalies with the help of the automated camera calibration, enabling instant camera pose estimation of traffic agents onto the road/intersection surface. The complete system achieves leading anomaly detection results on public datasets and it is deployed and validated on the roads of Helmond town in The Netherlands. Understanding the behaviour of traffic participants within the geo‐spatial context of road/intersection topology is a vital prerequisite for any smart ITS application. This article presents a video‐based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The system solves the challenge of geo‐spatial mapping of captured visual data onto the road/intersection topology by semantic analysis of aerial data. Additionally, the automated camera calibration component enables instant camera pose estimation to map traffic agents onto the road/intersection surface accurately. A novel aspect is approaching the anomaly detection problem by AI analysis of both the spatio‐temporal visual clues and the geo‐spatial trajectories for all type of traffic participants, such as pedestrians, bicyclists, and vehicles. This enables recognition of anomalies related to either traffic‐rule violations, for example, jaywalking, improper turns, zig‐zag driving, unlawful stops, or behavioural anomalies: littering, accidents, falling, vandalism, violence, infrastructure collapse etc. The method achieves leading anomaly detection results on benchmark datasets World Cup 2014, UCF‐Crime, XD‐Violence, and ShanghaiTech. All the obtained results are streamed and rendered in real‐time by the developed TGX digital twin visualizer. The complete system has been deployed and validated on the roads of Helmond town in The Netherlands.

Intelligent Traffic Monitoring with Hybrid AI

A Study of Situational Reasoning for Traffic Understanding

From Time to Space: Automatic Annotation of Unmarked Traffic Scene Based on Trajectory Data.

Artificial Intelligence Enabled Traffic Monitoring System

Geo‐spatial traffic behaviour analysis and anomaly detection for ITS applications

Towards an Edge Intelligence-Based Traffic Monitoring System

Application of Artificial Intelligence in Traffic Control System of Non-autonomous Vehicles at Signalized Road Intersection.

Artificial Intelligence-Based Real-Time Traffic Management

Open-ti: open traffic intelligence with augmented language model

A Framework for Knowledge Management and Automated Reasoning Applied on Intelligent Transport Systems

Artificial Hummingbird Optimization Algorithm With Hierarchical Deep Learning for Traffic Management in Intelligent Transportation Systems

An AI-Augmented Kalman Filter Approach to Monitoring Network Traffic Matrix

Intelligent Traffic Congestion Control Using Black Widow Optimization with Hybrid Deep Learning on Smart City Environment

AI Based Traffic and Automobile Monitoring System

Simulation-based Scenario Generation for Robust Hybrid AI for Autonomy

An Information Fusion Approach to Intelligent Traffic Signal Control Using the Joint Methods of Multiagent Reinforcement Learning and Artificial Intelligence of Things

An Ontological Approach to the Detection of Anomalies in Vehicular Ad Hoc Networks

Distributed Situation Assessment for Traffic Emergent Events

AtoMixer: Atom-based Interactive Visual Exploration of Traffic Surveillance Data.

AI Perspectives in Smart Cities and Communities to Enable Road Vehicle Automation and Smart Traffic Control