Abstract:Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.

Riding feeling recognition based on multi-head self-attention LSTM for driverless automobile

Recognition of Riding Feeling from EEG Based on Neural Network

Driver Identification Based on Gear Shift Events and Attention-Based Bidirectional Long Short-Term Memory for Manual Transmission System

A Hybrid Approach for Driving Behavior Recognition: Integration of CNN and Transformer-Encoder with EEG Data

EEG classification of driver mental states by deep learning

Driver Emotion Recognition Involving Multimodal Signals: Electrophysiological Response, Nasal-Tip Temperature, and Vehicle Behavior

EEG and ECG-Based Multi-Sensor Fusion Computing for Real-Time Fatigue Driving Recognition Based on Feedback Mechanism

Machine Learning Based Driver Emotion Monitoring for Vehicular IoT

E-Key: an EEG-Based Biometric Authentication and Driving Fatigue Detection System

EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks

Attention-Guided Multi-scale Convolutional Neural Network for Driving Fatigue Detection

EEG-based emotion recognition for road accidents in a simulated driving environment

CEEMDAN Fuzzy Entropy Based Fatigue Driving Detection Using Single-Channel EEG

Driver Emotion Recognition with a Hybrid Attentional Multimodal Fusion Framework

Electroencephalogram-Based Driver Emotional State Detection with Manifold Learning

Passenger hazard perception based on EEG signals for highly automated driving vehicles

Driver Distraction Detection Using Bidirectional Long Short-Term Network Based on Multiscale Entropy of EEG

Driver Multi-task Emotion Recognition Network Based on Multi-modal Facial Video Analysis

Recognition of driver’s mental workload based on physiological signals, a comparative study

Real-time driving risk prediction using a self-attention-based bidirectional long short-term memory network based on multi-source data

EEG-Based Spatio-Temporal Convolutional Neural Network for Driver Fatigue Evaluation