Abstract:In recent years, deep learning-based driver fatigue detection algorithms have been increasingly used. However, traditional fatigue detection algorithms cannot effectively correlate contextual information of image frames. They perform better in individual image frames. Also, the accuracy and robustness of these algorithms are limited because they only consider particular frames. Therefore, a fatigue detection method based on integrated facial features and Gate Recurrent Unit (GRU) judgment neural network is proposed in this paper. We use a neural network including a GRU layer to efficiently distinguish the contextual information present in multiple image frames arranged in chronological order. Besides, we designed a multi-task convolutional neural network (MTCNN) model to extract comprehensive facial features. After obtaining the facial feature points' positions, we can calculate the aspect ratio between the upper and lower eyelids, the upper and lower lips, and the eyebrows to the chin. In addition to the above three features, we can also obtain the subject's three head pose angles by comparing the facial features with the typical 3D face model. Finally, we input the change curves of 6 features in 20 consecutive frames into the judgment network to learn the change rule and create a judgment network. After the learning is completed, the judgment network model will judge the six feature curves in the newly input 20 frames in real-time and output the driver's fatigue status. This fatigue detection method can take a real-time detection at 55 FPS on the workstation platform (TensorFlow 2.3.0, RTX2070s). On the Nvidia Jetson Xavier AGX embedded platform (TensorFlow lite, ARM 8-cores CPU), the method can take a real-time detection at 26 FPS. The accuracy of this fatigue detection method can reach 97.47%.

Driver Fatigue Detection Based on Facial Key Points and LSTM.

Fatigue Driving Facial Feature Detection Based on Lightweight Convolutional Neural Network

Research on Fatigue Driving Detection Method Based on Lightweight Convolutional Neural Network

Driver fatigue detection based on comprehensive facial features and gated recurrent unit

Adaptive Driver Face Feature Fatigue Detection Algorithm Research

A New Algorithm Based on Cascaded Convolutional Neural Network for Driver Fatigue Detection

Researching Driver Fatigue Detection Using Video Technology

Real-Time Detection of Fatigue Driving Based on Face Recognition

Vision-Based Fatigue Driving Recognition Method Integrating Heart Rate and Facial Features

Research on Driver Fatigue State Detection Method Based on Deep Learning

Driver fatigue detection based on deeply-learned facial expression representation

A Fatigue Driving Detection Algorithm Based on Facial Multi-Feature Fusion

Remote Photoplethysmography and Motion Tracking Convolutional Neural Network with Bidirectional Long Short-Term Memory: Non-Invasive Fatigue Detection Method Based on Multi-Modal Fusion

Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences

Fatigue Driving Detection Based on Deep Learning and Multi-Index Fusion

A Driver Fatigue Recognition Algorithm Based on Spatio-Temporal Feature Sequence

Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN

Driver Emotion and Fatigue State Detection Based on Time Series Fusion

Real-time detection method of driver fatigue state based on deep learning of face video

A Practical Eye State Recognition Based Driver Fatigue Detection Method

Driver Fatigue Detection Based on Parallel Neural Network