Abstract:Traffic accident prediction in driving videos aims to provide an early warning of the accident occurrence, and supports the decision making of safe driving systems. Previous works usually concentrate on the spatial-temporal correlation of object-level context, while they do not fit the inherent long-tailed data distribution well and are vulnerable to severe environmental change. In this work, we propose a Cognitive Accident Prediction (CAP) method that explicitly leverages human-inspired cognition of text description on the visual observation and the driver attention to facilitate model training. In particular, the text description provides a dense semantic description guidance for the primary context of the traffic scene, while the driver attention provides a traction to focus on the critical region closely correlating with safe driving. CAP is formulated by an attentive text-to-vision shift fusion module, an attentive scene context transfer module, and the driver attention guided accident prediction module. We leverage the attention mechanism in these modules to explore the core semantic cues for accident prediction. In order to train CAP, we extend an existing self-collected DADA-2000 dataset (with annotated driver attention for each frame) with further factual text descriptions for the visual observations before the accidents. Besides, we construct a new large-scale benchmark consisting of 11,727 in-the-wild accident videos with over 2.19 million frames (named as CAP-DATA) together with labeled fact-effect-reason-introspection description and temporal accident frame label. Based on extensive experiments, the superiority of CAP is validated compared with state-of-the-art approaches. The code, CAP-DATA, and all results will be released in \url{<a class="link-external link-https" href="https://github.com/JWFanggit/LOTVS-CAP" rel="external noopener nofollow">this https URL</a>}.

Improved Dynamic Spatial-Temporal Attention Network for Early Anticipation of Traffic Accidents

A Dynamic Spatial-temporal Attention Network for Early Anticipation of Traffic Accidents

CRASH: Crash Recognition and Anticipation System Harnessing with Context-Aware and Temporal Focus Attentions

Dynamic Attention Augmented Graph Network for Video Accident Anticipation

Real-time Accident Anticipation for Autonomous Driving Through Monocular Depth-Enhanced 3D Modeling

Cognitive Traffic Accident Anticipation

Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learning

STDA: Spatio-Temporal Dual-Encoder Network Incorporating Driver Attention to Predict Driver Behaviors Under Safety-Critical Scenarios

Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark

DADA: Driver Attention Prediction in Driving Accident Scenarios

DADA-2000: Can Driving Accident Be Predicted by Driver Attentionƒ Analyzed by A Benchmark

DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark

Unsupervised Traffic Accident Detection in First-Person Videos

Dynamic Spatial–Temporal Self-Attention Network for Traffic Flow Prediction

Augmenting Ego-Vehicle for Traffic Near-Miss and Accident Classification Dataset using Manipulating Conditional Style Translation

Driving Video Fixation Prediction Model Via Spatio-Temporal Networks and Attention Gates

Deep Spatio-Temporal Graph Convolutional Network for Traffic Accident Prediction

A Deep Learning Framework Based on Spatio-Temporal Attention Mechanism for Traffic Prediction.

Traffic Accident Prediction Based on Deep Spatio-Temporal Analysis.

SDCAE: Stack Denoising Convolutional Autoencoder Model for Accident Risk Prediction Via Traffic Big Data