Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

Multi-Sensor Based Perception Network for Vehicle Driving Assistance

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

A Novel Multi-Sensor Environmental Perception Method Using Low-Rank Representation and a Particle Filter for Vehicle Reversing Safety

A Multi-view 3D Vehicle Detection Method Based On Novel 3D Proposal Generation Method

A Multimodal Data-Driven Approach for Driving Risk Assessment

A Detachable and Expansible Multisensor Data Fusion Model for Perception in Level 3 Autonomous Driving System

Comprehensive understanding of random telegraph noise with

Design and Research of Multi-sensor Detection System for Internet of Vehicles

Sensor-Based Environmental Perception Technology for Intelligent Vehicles

REAL-SAP: Real-time Evidence Aware Liable Safety Assessment for Perception in Autonomous Driving

Real-Time Monocular Joint Perception Network for Autonomous Driving

Multi-sensor fusion algorithm in cooperative vehicle-infrastructure system for blind spot warning

V2X-AHD:Vehicle-to-Everything Cooperation Perception via Asymmetric Heterogenous Distillation Network

A Multi-Task Network Based on Dual-Neck Structure for Autonomous Driving Perception

Multitarget Vehicle Tracking and Motion State Estimation Using a Novel Driving Environment Perception System of Intelligent Vehicles

Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

Confidence Evaluation for Machine Learning Schemes in Vehicular Sensor Networks

Multi-Modal Sensor Fusion-Based Deep Neural Network for End-to-End Autonomous Driving With Scene Understanding

Human-vehicle Cooperative Visual Perception for Autonomous Driving under Complex Road and Traffic Scenarios

Reinforce Learning-Based Collision Avoidance in Network Assisted Automated Driving