Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Dilated Light-Head R-CNN using tri-center loss for driving behavior recognition

Driver Action Recognition Using Deformable and Dilated Faster R-CNN with Optimized Region Proposals

Driving Behavior Recognition Algorithm Combining Attention Mechanism and Lightweight Network

Long Short‐term Memory and Convolutional Neural Network for Abnormal Driving Behaviour Recognition

Driving Behaviour Recognition from Still Images by Using Multi-Stream Fusion CNN.

Recongnition of Distracted Driving Behavior Based on Improved Bi-LSTM Model and Attention Mechanism

Vehicle Behavior Recognition using Multi-Stream 3D Convolutional Neural Network

An abnormal driving behavior recognition algorithm based on the temporal convolutional network and soft thresholding

Application of Deep Learning Methods in Driver Behavior Recognition

Driver Detection Based on Deep Learning

Driver behavior recognition based on deep convolutional neural networks

Learning Driver-Irrelevant Features for Generalizable Driver Behavior Recognition

A Multimodal Data-Driven Approach for Driving Risk Assessment

Driving Behaviour Characterisation by Using Phase‐space Reconstruction and Pre‐trained Convolutional Neural Network

A driver’s dangerous driving behavior detection method based on the YOLOV5-CA network

Recognition method of abnormal driving behavior using the bidirectional gated recurrent unit and convolutional neural network

Image sequence recognition method for driving behavior based on improved LRCN model

RFDANet: an FMCW and TOF radar fusion approach for driver activity recognition using multi-level attention based CNN and LSTM network

Pose-guided Model for Driving Behavior Recognition Using Keypoint Action Learning

A Hybrid Approach for Driving Behavior Recognition: Integration of CNN and Transformer-Encoder with EEG Data

Multi-CNN and Decision Tree Based Driving Behavior Evaluation