Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Impaired Driving Detection Based on Deep Convolutional Neural Network Using Multimodal Sensor Data

Fatigue Driving Facial Feature Detection Based on Lightweight Convolutional Neural Network

<italic>DetectDUI</italic>: An In-Car Detection System for Drink Driving and BACs

Video-Based Abnormal Driving Behavior Detection Via Deep Learning Fusions

A Multimodal Data-Driven Approach for Driving Risk Assessment

Vehicle detection method based on deep learning and multi-layer feature fusion

HSDDD: A Hybrid Scheme for the Detection of Distracted Driving through Fusion of Deep Learning and Handcrafted Features

Mfdd: Multi-scale Attention Fatigue and Distracted Driving Detector Based on Facial Features

Convolutional neural network with transfer learning approach for detection of unfavorable driving state using phase coherence image

Fine-Grained Detection of Driver Distraction Based on Neural Architecture Search

Detecting Abnormal Driving Behaviors By Smartphone Sensors Based On Multi-Feature Convolutional Neural Network

FMDNet: Feature-Attention-Embedding-Based Multimodal-Fusion Driving-Behavior-Classification Network

Object Detection Using Multi-Sensor Fusion Based on Deep Learning

Perceiving Driving Hazards in a Data-fusion Way Using Multi-Modal Net and Semantic Driving Trajectory

Expressway Traffic Incident Detection Using a Deep Learning Approach Based on Spatiotemporal Features with Multilevel Fusion

Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection

Towards Sustainable Safe Driving: A Multimodal Fusion Method for Risk Level Recognition in Distracted Driving Status

Driver Behavior Recognition via Interwoven Deep Convolutional Neural Nets With Multi-Stream Inputs

Multimodal Features Fusion for Driver Fatigue Detection Based on CNN-GTN Learning

Driver behavior detection and classification using deep convolutional neural networks

Processing and Integration of Multimodal Image Data Supporting the Detection of Behaviors Related to Reduced Concentration Level of Motor Vehicle Users