Abstract:Traffic accident prediction in driving videos aims to provide an early warning of the accident occurrence, and supports the decision making of safe driving systems. Previous works usually concentrate on the spatial-temporal correlation of object-level context, while they do not fit the inherent long-tailed data distribution well and are vulnerable to severe environmental change. In this work, we propose a Cognitive Accident Prediction (CAP) method that explicitly leverages human-inspired cognition of text description on the visual observation and the driver attention to facilitate model training. In particular, the text description provides a dense semantic description guidance for the primary context of the traffic scene, while the driver attention provides a traction to focus on the critical region closely correlating with safe driving. CAP is formulated by an attentive text-to-vision shift fusion module, an attentive scene context transfer module, and the driver attention guided accident prediction module. We leverage the attention mechanism in these modules to explore the core semantic cues for accident prediction. In order to train CAP, we extend an existing self-collected DADA-2000 dataset (with annotated driver attention for each frame) with further factual text descriptions for the visual observations before the accidents. Besides, we construct a new large-scale benchmark consisting of 11,727 in-the-wild accident videos with over 2.19 million frames (named as CAP-DATA) together with labeled fact-effect-reason-introspection description and temporal accident frame label. Based on extensive experiments, the superiority of CAP is validated compared with state-of-the-art approaches. The code, CAP-DATA, and all results will be released in \url{<a class="link-external link-https" href="https://github.com/JWFanggit/LOTVS-CAP" rel="external noopener nofollow">this https URL</a>}.

Driving Accident Detection by Self-Supervised Adversarial Appearance-Motion Prediction in First-Person Videos

Unsupervised Traffic Accident Detection in First-Person Videos

Traffic Accident Detection Via Self-Supervised Consistency Learning in Driving Scenarios

A Memory-Augmented Multi-Task Collaborative Framework for Unsupervised Traffic Accident Detection in Driving Videos

CRASH: Crash Recognition and Anticipation System Harnessing with Context-Aware and Temporal Focus Attentions

Improved Dynamic Spatial-Temporal Attention Network for Early Anticipation of Traffic Accidents

Computer Vision based Accident Detection for Autonomous Vehicles

A Dynamic Spatial-temporal Attention Network for Early Anticipation of Traffic Accidents

Real-time accident anticipation for autonomous driving through monocular depth-enhanced 3D modeling

Review of Accident Detection Methods Using Dashcam Videos for Autonomous Driving Vehicles

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

Efficient Traffic Accident Warning Based on Unsupervised Prediction Framework

GenAD: Generalized Predictive Model for Autonomous Driving

Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving

Adaptive Visual Interaction Based Multi-Target Future State Prediction For Autonomous Driving Vehicles

Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving

Failure Prediction for Autonomous Driving

Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoring

Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark