Abstract:Traffic command and scheduling are the core monitoring aspects of railway transportation. Detecting the fatigued state of dispatchers is, therefore, of great significance to ensure the safety of railway operations. In this paper, we present a multi-feature fatigue detection method based on key points of the human face and body posture. Considering unfavorable factors such as facial occlusion and angle changes that have limited single-feature fatigue state detection methods, we developed our model based on the fusion of body postures and facial features for better accuracy. Using facial key points and eye features, we calculate the percentage of eye closure that accounts for more than 80% of the time duration, as well as blinking and yawning frequency, and we analyze fatigue behaviors, such as yawning, a bowed head (that could indicate sleep state), and lying down on a table, using a behavior recognition algorithm. We fuse five facial features and behavioral postures to comprehensively determine the fatigue state of dispatchers. The results show that on the 300 W dataset, as well as a hand-crafted dataset, the inference time of the improved facial key point detection algorithm based on the retina–face model was 100 ms and that the normalized average error (NME) was 3.58. On our own dataset, the classification accuracy based the an Bi-LSTM-SVM adaptive enhancement algorithm model reached 97%. Video data of volunteers who carried out scheduling operations in the simulation laboratory were used for our experiments, and our multi-feature fusion fatigue detection algorithm showed an accuracy rate of 96.30% and a recall rate of 96.30% in fatigue classification, both of which were higher than those of existing single-feature detection methods. Our multi-feature fatigue detection method offers a potential solution for fatigue level classification in vital areas of the industry, such as in railway transportation.

Air Traffic Controller Fatigue Detection Based on Facial and Vocal Features Using Long Short-Term Memory

On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features

Air Traffic Controller Fatigue Detection by Applying a Dual-Stream Convolutional Neural Network to the Fusion of Radiotelephony and Facial Data

Fatigue Detection of Air Traffic Controllers Through Their Eye Movements

Researching Driver Fatigue Detection Using Video Technology

Controller Fatigue State Detection Based on ES-DFNN

Research on Railway Dispatcher Fatigue Detection Method Based on Deep Learning with Multi-Feature Fusion

Fatigue Detection in Air Traffic Controllers: A Comprehensive Review

A High-Efficiency Fatigued Speech Feature Selection Method for Air Traffic Controllers Based on Improved Compressed Sensing

Research on Driver Fatigue Detection

Modeling and analysis of fatigue detection with multi-channel data fusion

A Novel Temporal Adaptive Fuzzy Neural Network for Facial Feature Based Fatigue Assessment

Spatial-Temporal Feature Representation Learning for Facial Fatigue Detection.

Nonintrusive Driver Fatigue Detection

Real-Time Detection of Fatigue Driving Based on Face Recognition

Multi-parameter Fusion Driver Fatigue Detection Method Based on Facial Fatigue Features

Detection of mental fatigue state using heart rate variability and eye metrics during simulated flight

Fatigue Detection Based on Fast Facial Feature Analysis.

Train Driver Fatigue Detection Using Eye Feature Vector and Support Vector Machine

Driver fatigue detection based on deeply-learned facial expression representation

Speech Fatigue Detection Based on Deep Learning