Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Identifying High Risk Driving Scenarios Utilizing a CNN-LSTM Analysis Approach.

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Real-Time Crash Risk Prediction using Long Short-Term Memory Recurrent Neural Network

Driving risk assessment under the connected vehicle environment: A CNN-LSTM modeling approach

A Multimodal Data-Driven Approach for Driving Risk Assessment

Long Short‐term Memory and Convolutional Neural Network for Abnormal Driving Behaviour Recognition

A Lightweight VK-Net Based on Motion Profiles for Hazardous Driving Scenario Identification

Vehicle Forward Collision Warning Based Upon Low Frequency Video Data: A Hybrid Deep Learning Modeling Approach

Moving vehicle tracking and scene understanding: A hybrid approach

LSTM $+$ Transformer Real-Time Crash Risk Evaluation Using Traffic Flow and Risky Driving Behavior Data

Predicting Hazardous Driving Events Using Multi-Modal Deep Learning Based on Video Motion Profile and Kinematics Data

Real-Time Driving Behavior Identification Based on Multi-Source Data Fusion

Risky Action Recognition in Lane Change Video Clips using Deep Spatiotemporal Networks with Segmentation Mask Transfer

RFDANet: an FMCW and TOF radar fusion approach for driver activity recognition using multi-level attention based CNN and LSTM network

Real-time crash risk prediction on arterials based on LSTM-CNN

Prediction of traffic accident risk based on vehicle trajectory data.

Real-Time Run-Off-Road Risk Prediction Based on Deep Learning Sequence Forecasting Approach

Real-time video surveillance on highways using combination of extended Kalman Filter and deep reinforcement learning

Real-time driving risk prediction using a self-attention-based bidirectional long short-term memory network based on multi-source data

Video-based Driver Action Recognition Via Hybrid Spatial–temporal Deep Learning Framework

Driving Intention Identification Based on Long Short-Term Memory Neural Network