Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

ThirdEye: Attention Maps for Safe Autonomous Driving Systems

A Multimodal Data-Driven Approach for Driving Risk Assessment

Classification of Safety Driver Attention During Autonomous Vehicle Operation

Human Visual Attention Prediction Boosts Learning & Performance of Autonomous Driving Agents

Online Collision Risk Estimation via Monocular Depth-Aware Object Detectors and Fuzzy Inference

Runtime Stealthy Perception Attacks against DNN-based Adaptive Cruise Control Systems

A-Eye: Driving with the Eyes of AI for Corner Case Generation

Failure Prediction for Autonomous Driving

A Gaze Data-based Comparative Study to Build a Trustworthy Human-AI Collaboration in Crash Anticipation

AIMSafe: EEG-Based Driver Behavior Understanding Via Attention and Incremental Learning Mechanisms

Predicting Safety Misbehaviours in Autonomous Driving Systems using Uncertainty Quantification

Visual attention for behavioral cloning in autonomous driving

Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoring

CRASH: Crash Recognition and Anticipation System Harnessing with Context-Aware and Temporal Focus Attentions

Modeling Drivers' Risk Perception via Attention to Improve Driving Assistance

Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles

Recent advancements in driver's attention prediction

Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated Driving

When, Where and How Does It Fail? A Spatial-Temporal Visual Analytics Approach for Interpretable Object Detection in Autonomous Driving.

DeepTake: Prediction of Driver Takeover Behavior using Multimodal Data

Real-time Accident Anticipation for Autonomous Driving Through Monocular Depth-Enhanced 3D Modeling