Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Attention-Driven Driving Maneuver Detection System.

Driving Maneuver Early Detection Via Sequence Learning from Vehicle Signals and Video Images.

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

A Multimodal Data-Driven Approach for Driving Risk Assessment

FDAN: Fuzzy deep attention networks for driver behavior recognition

DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis

Driving Maneuvers Prediction Based on Cognition-Driven and Data-Driven Method.

E-DNet: An End-to-End Dual-Branch Network for Driver Steering Intention Detection

M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving

Multisource Adaption for Driver Attention Prediction in Arbitrary Driving Scenes

Deep learning approach for accurate and stable recognition of driver's lateral intentions using naturalistic driving data

Driver Maneuver Identification with Multi-Representation Learning and Meta Model Update Designs

Multi-level Attention Fusion for Multimodal Driving Maneuver Recognition

All in One Network for Driver Attention Monitoring

STDA: Spatio-Temporal Dual-Encoder Network Incorporating Driver Attention to Predict Driver Behaviors Under Safety-Critical Scenarios

Enhancing Road Safety: Deep Learning-Based Intelligent Driver Drowsiness Detection for Advanced Driver-Assistance Systems

Double Attention Convolutional Neural Network for Driver Action Recognition

Attention-Based Deep Driving Model for Autonomous Vehicles with Surround-View Cameras

Analysis of Distracted Driving Detection Based on Deep Learning Human Posture.

Recognition of Lane Changing Maneuvers for Vehicle Driving Safety

A Novel Method for Unsafe Driving States Detection