Abstract:Traffic command and scheduling are the core monitoring aspects of railway transportation. Detecting the fatigued state of dispatchers is, therefore, of great significance to ensure the safety of railway operations. In this paper, we present a multi-feature fatigue detection method based on key points of the human face and body posture. Considering unfavorable factors such as facial occlusion and angle changes that have limited single-feature fatigue state detection methods, we developed our model based on the fusion of body postures and facial features for better accuracy. Using facial key points and eye features, we calculate the percentage of eye closure that accounts for more than 80% of the time duration, as well as blinking and yawning frequency, and we analyze fatigue behaviors, such as yawning, a bowed head (that could indicate sleep state), and lying down on a table, using a behavior recognition algorithm. We fuse five facial features and behavioral postures to comprehensively determine the fatigue state of dispatchers. The results show that on the 300 W dataset, as well as a hand-crafted dataset, the inference time of the improved facial key point detection algorithm based on the retina–face model was 100 ms and that the normalized average error (NME) was 3.58. On our own dataset, the classification accuracy based the an Bi-LSTM-SVM adaptive enhancement algorithm model reached 97%. Video data of volunteers who carried out scheduling operations in the simulation laboratory were used for our experiments, and our multi-feature fusion fatigue detection algorithm showed an accuracy rate of 96.30% and a recall rate of 96.30% in fatigue classification, both of which were higher than those of existing single-feature detection methods. Our multi-feature fatigue detection method offers a potential solution for fatigue level classification in vital areas of the industry, such as in railway transportation.

Speech Fatigue Detection Based on Deep Learning

Detecting Sports Fatigue from Speech by Support Vector Machine

Sports Fatigue Detection Based on Deep Learning

Intelligent Emotion Detection Method Based on Deep Learning in Medical and Health Data

Research on Railway Dispatcher Fatigue Detection Method Based on Deep Learning with Multi-Feature Fusion

Air Traffic Controller Fatigue Detection Based on Facial and Vocal Features Using Long Short-Term Memory

Identification of mental fatigue in language comprehension tasks based on EEG and deep learning

Method of applying speech multi-features to detect driver fatigue

Speech-Based Five Features Extraction and Hardware Trade-Off Design for Mental Fatigue Monitoring

Data-driven learning fatigue detection system: A multimodal fusion approach of ECG (electrocardiogram) and video signals

Speech Fatigue Recognition Under Small Samples Based on Generative Adversarial Networks and BLSTM

Detecting Fatigue Status of Pilots Based on Deep Learning Network Using EEG Signals

Driver fatigue detection based on deeply-learned facial expression representation

Recognition of Fatigue Status of Pilots Based on Deep Contractive Sparse Auto-Encoding Network

Research of detecting fatigue from speech by PNN

Fatigue Detection Technology for Online Learning

Machine vision detection to daily facial fatigue with a nonlocal 3D attention network

Fatigue driving detection method based on Time-Space-Frequency features of multimodal signals

Recent Advances in Fatigue Detection Algorithm Based on EEG

Deep Learning for Fatigue Estimation on the Basis of Multimodal Human-Machine Interactions

Medical intelligence using PPG signals and hybrid learning at the edge to detect fatigue in physical activities