Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Detecting Unsafe Driving Patterns Using Discriminative Learning

A Hybrid Deep Learning Approach for Driver Anomalous Lane Changing Identification

A Multimodal Data-Driven Approach for Driving Risk Assessment

SLAFusion: Attention fusion based on SAX and LSTM for dangerous driving behavior detection.

Symbolic Aggregate Approximation Based Data Fusion Model for Dangerous Driving Behavior Detection

Mid-term prediction of at-fault crash driver frequency using fusion deep learning with city-level traffic violation data

An Attention-Based Car-Following Model Based on Fused data

FedDAF: Federated Deep Attention Fusion for Dangerous Driving Behavior Detection

Temporal Information Fusion Network for Driving Behavior Prediction.

MM-LMF: A Low-Rank Multimodal Fusion Dangerous Driving Behavior Recognition Method Based on FMCW Signals

Towards Sustainable Safe Driving: A Multimodal Fusion Method for Risk Level Recognition in Distracted Driving Status

A Hybrid Method Combining Markov Prediction and Fuzzy Classification for Driving Condition Recognition.

Real-Time Driving Behavior Identification Based on Multi-Source Data Fusion

Efficient Traffic Accident Warning Based on Unsupervised Prediction Framework

A Machine Learning Framework for Automated Accident Detection Based on Multimodal Sensors in Cars

Discriminative Transfer Learning for Driving Pattern Recognition in Unlabeled Scenes

Driving Safety Monitoring Using Semisupervised Learning on Time Series Data

Driver intention prediction based on multi-dimensional cross-modality information interaction

A maneuver indicator and ensemble learning-based risky driver recognition approach for highway merging areas

DSDCLA: driving style detection via hybrid CNN-LSTM with multi-level attention fusion