Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Driver intention prediction based on multi-dimensional cross-modality information interaction

Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoring

Spatiotemporal Feature Enhancement Aids the Driving Intention Inference of Intelligent Vehicles

Looking Inside Out: Anticipating Driver Intent From Videos

A Multimodal Data-Driven Approach for Driving Risk Assessment

Driver Intent-Based Intersection Autonomous Driving Collision Avoidance Reinforcement Learning Algorithm

Adaptive Visual Interaction Based Multi-Target Future State Prediction For Autonomous Driving Vehicles

Real-time driving risk prediction using a self-attention-based bidirectional long short-term memory network based on multi-source data

Driver lane change intention prediction based on topological graph constructed by driver behaviors and traffic context for human-machine co-driving system

Convolutional neural network-based intention forecasting and lane change path predicting of the human driver

A driving intention prediction method based on hidden Markov model for autonomous driving

A comprehensive lateral motion prediction method of surrounding vehicles integrating driver intention prediction and vehicle behavior recognition

Map-Adaptive Multimodal Trajectory Prediction via Intention-Aware Unimodal Trajectory Predictors

Deep learning approach for unified recognition of driver speed and lateral intentions using naturalistic driving data

Driver Behavior Recognition via Interwoven Deep Convolutional Neural Nets With Multi-Stream Inputs

Driving Behavior Prediction Considering Cognitive Prior and Driving Context

Toward Driver Intention Prediction for Intelligent Vehicles: A Deep Learning Approach

DeepInteraction++: Multi-Modality Interaction for Autonomous Driving

Multi-Interaction Trajectory Prediction Method With Serial Attention Patterns for Intelligent Vehicles

Temporal Information Fusion Network for Driving Behavior Prediction.

Multi-Modal Vehicle Trajectory Prediction by Collaborative Learning of Lane Orientation, Vehicle Interaction, and Intention