Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Ensemble of Randomized Neural Network and Boosted Trees for Eye-Tracking-Based Driver Situation Awareness Recognition and Interpretation.

Driving Behaviour Style Study with a Hybrid Deep Learning Framework Based on GPS Data

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated Driving

A Multimodal Data-Driven Approach for Driving Risk Assessment

Evaluation of Driver’s Situation Awareness in Freeway Exit Using Backpropagation Neural Network

Multimodal Sensing and Computational Intelligence for Situation Awareness Classification in Autonomous Driving

Towards Context-Aware Modeling of Situation Awareness in Conditionally Automated Driving

An Interpretable Prediction Framework for Multi-Class Situational Awareness in Conditionally Automated Driving

Sensor-based Risk Perception Ability Network Design for Drivers in Snow and Ice Environmental Freeway: a Deep Learning and Rough Sets Approach

Decision-Making for Autonomous Vehicles with Interaction-Aware Behavioral Prediction and Social-Attention Neural Network

Driver's Visual Fixation Attention Prediction in Dynamic Scenes Using Hybrid Neural Networks

When Do Drivers Concentrate? Attention-based Driver Behavior Modeling With Deep Reinforcement Learning

How Do Drivers Allocate Their Potential Attention? Driving Fixation Prediction via Convolutional Neural Networks

A Regression Method With Subnetwork Neurons for Vigilance Estimation Using EOG and EEG

Deep Learning-Enhanced Environment Perception for Autonomous Driving: MDNet with CSP-DarkNet53

Faster Single Model Vigilance Detection Based on Deep Learning

Multisource Adaption for Driver Attention Prediction in Arbitrary Driving Scenes

Real-time driving risk assessment using deep learning with XGBoost

A Recognition Method of Aggressive Driving Behavior Based on Ensemble Learning

An Eye-Fixation Related Electroencephalography Technique for Predicting Situation Awareness: Implications for Driver State Monitoring Systems