Abstract:The detection of unfavorable driving states (UDS) of drivers based on electroencephalogram (EEG) measures has received continuous attention from extensive scholars on account of directly reflecting brain neural activity with high temporal resolution and low risk of being deceived. However, the existing EEG-based driver UDS detection methods involve limited exploration of the functional connectivity patterns and interaction relationships within the brain network. Therefore, there is still room for improvement in the accuracy of detection. In this project, we propose three pretrained convolutional neural network (CNN)-based automatic detection frameworks for UDS of drivers with 30-channel EEG signals. The frameworks are investigated by adjusting the learning rate and choosing the optimization solver, etc. Two different conditions of driving experiments are performed, collecting EEG signals from sixteen subjects. The acquired 1-dimensional 30-channel EEG signals are converted into 2-dimensional matrices by the Granger causality (GC) method to form the functional connectivity graphs of the brain (FCGB). Then, the FCGB are fed into pretrained deep learning models that employed transfer learning strategy for feature extraction and judgment of different EEG signal types. Furthermore, we adopt two visualization interpretability techniques, named, activation visualization and gradient-weighted class activation mapping (Grad-CAM) for better visualizing and understanding the predictions of the pretrained models after fine-tuning. The experimental outcomes show that Resnet 18 model yields the highest average recognition accuracy of 90% using the rmsprop optimizer with a learning rate of 1e − 3. The overall outcomes suggest that cooperating of biologically inspired functional connectivity graphs of the brain and pretrained transfer learning algorithms is a prospective approach in reducing the rate of major traffic accidents caused by driver unfavorable driving states.

Temporal Difference Enhancement Network for Driving Behavior Recognition

An abnormal driving behavior recognition algorithm based on the temporal convolutional network and soft thresholding

FDAN: Fuzzy deep attention networks for driver behavior recognition

Driver Distraction Recognition Based on CBAM Attention Mechanism.

Driving Behavior Recognition Algorithm Combining Attention Mechanism and Lightweight Network

Temporal Information Fusion Network for Driving Behavior Prediction.

Recongnition of Distracted Driving Behavior Based on Improved Bi-LSTM Model and Attention Mechanism

E-DNet: An End-to-End Dual-Branch Network for Driver Steering Intention Detection

Two-Stream (2+1)D CNN Based on Frame Difference Attention for Driver Behavior Recognition

RFDANet: an FMCW and TOF radar fusion approach for driver activity recognition using multi-level attention based CNN and LSTM network

Context-Aware Attention Encoder-Decoder Network for Connected Heavy-Duty Vehicle Aggressive Driving Identification under Naturalistic Driving Conditions

Improving automatic detection of driver fatigue and distraction using machine learning

STDA: Spatio-Temporal Dual-Encoder Network Incorporating Driver Attention to Predict Driver Behaviors Under Safety-Critical Scenarios

A Transfer Learning-Based CNN Deep Learning Model for Unfavorable Driving State Recognition

Drive-Net: Convolutional Network for Driver Distraction Detection

Driving Fatigue Detection Based on Tiny Frequency-Temporal Network from EEG Signals

Robust Two-Stream Multi-Features Network for Driver Drowsiness Detection

A Hybrid Approach for Driving Behavior Recognition: Integration of CNN and Transformer-Encoder with EEG Data

A TSK-Type Convolutional Recurrent Fuzzy Network for Predicting Driving Fatigue

End-to-End Driving Activities and Secondary Tasks Recognition Using Deep Convolutional Neural Network and Transfer Learning

A driver’s dangerous driving behavior detection method based on the YOLOV5-CA network