Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.

A Contrastive-Learning-Based Method for Alert-Scene Categorization

An Image-based Approach of Task-driven Driving Scene Categorization

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos

A Multimodal Data-Driven Approach for Driving Risk Assessment

Contrastive Attention for Video Anomaly Detection

Cognitive Traffic Accident Anticipation

Driver Anomaly Detection: A Dataset and Contrastive Learning Approach

Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detection

DADA: Driver Attention Prediction in Driving Accident Scenarios

Modeling Drivers' Risk Perception via Attention to Improve Driving Assistance

Multisource Adaption for Driver Attention Prediction in Arbitrary Driving Scenes

Abnormal Event Detection Using Deep Contrastive Learning for Intelligent Video Surveillance System

Video Class Agnostic Segmentation with Contrastive Learning for Autonomous Driving

Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive Learning

Driving Cognitive Alertness Detecting Using Evoked Multimodal Physiological Signals Based on Uncertain Self-Supervised Learning

Ego-Vehicle Action Recognition based on Semi-Supervised Contrastive Learning

L-TLA: A Lightweight Driver Distraction Detection Method Based on Three-Level Attention Mechanisms

Directed Mix Contrast for Lidar Point Cloud Segmentation

SegContrast: 3D Point Cloud Feature Representation Learning Through Self-Supervised Segment Discrimination

Multimodal-Enhanced Objectness Learner for Corner Case Detection in Autonomous Driving