Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

SafeDrive: Knowledge- and Data-Driven Risk-Sensitive Decision-Making for Autonomous Vehicles with Large Language Models

A Multimodal Data-Driven Approach for Driving Risk Assessment

DriveLLM: Charting the Path Toward Full Autonomous Driving with Large Language Models

Decision Making of Autonomous Vehicles in Lane Change Scenarios: Deep Reinforcement Learning Approaches with Risk Awareness

Empowering Autonomous Driving with Large Language Models: A Safety Perspective

CALMM-Drive: Confidence-Aware Autonomous Driving with Large Multimodal Model

DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models

Receive, Reason, and React: Drive as You Say, With Large Language Models in Autonomous Vehicles

Deep Reinforcement Learning Enabled Decision-Making for Autonomous Driving at Intersections

Risk-anticipatory autonomous driving strategies considering vehicles' weights, based on hierarchical deep reinforcement learning

Safedrive Dreamer: Navigating Safety–critical Scenarios in Autonomous Driving with World Models

Risk-informed decision-making and control strategies for autonomous vehicles in emergency situations

Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles

Continuous Decision-Making in Lane Changing and Overtaking Maneuvers for Unmanned Vehicles: A Risk-Aware Reinforcement Learning Approach with Task Decomposition

Deep Reinforcement Learning Based Decision-Making Strategy of Autonomous Vehicle in Highway Uncertain Driving Environments

Runtime Safety Assurance for Learning-enabled Control of Autonomous Driving Vehicles

World knowledge-enhanced Reasoning Using Instruction-guided Interactor in Autonomous Driving

SMART: A Decision-Making Framework with Multi-modality Fusion for Autonomous Driving Based on Reinforcement Learning.

Graph Structure-Based Implicit Risk Reasoning for Long-Tail Scenarios of Automated Driving

Personalized Autonomous Driving with Large Language Models: Field Experiments

An Integrated Decision and Motion Planning Framework for Automated Driving on Highway