Abstract:When executing complex missions in emergency scenarios, pilots' cognitive state may deteriorate, posing significant challenges to flight safety and mission execution. This paper proposes a cross-subject and cross-session early warning approach based on small-sample and multi-modal signals for predicting cognitive collapse state. We designs an experimental paradigm that has been demonstrated to induce cognitive collapse in 87 % of trials by analyzing questionnaire scores, physiological signals, and task performance. The extracted multi-modal features are fused and selected to construct the optimal feature set. Further, a two-step early warning method is introduced to identify critical slowing down and to predict tipping points by detecting the high cognitive workload state and classifying the corresponding signals into early warning signal (EWS) and normal state. The early warning of pilot's cognitive tipping point obtains the result that recall is 77.76 % , false alarm rate is 28.43 % , and the AUC of the warning method can reach 0.83. Our method shows better early warning performance compared with other classification models and has strong generalizability on other datasets. Note to Practitioners -The pilots' cognitive state is crucial for mission completion and flight safety. Previous studies mainly focused on assessing current cognitive state, and there is little research on early warning of cognitive collapse that occurs in emergency scenarios (e.g., faults of system and the sudden increase in flight missions). In particular, identification is not effective for small-sample and instable data. Therefore, this paper presents an early warning method for cognitive tipping points, which can provide timely warning of pilots' cognitive collapse and reduce human-caused risks in emergency scenarios.

A Method for Classification and Evaluation of Pilot’s Mental States Based on CNN

EEG-based analysis for pilots' at-risk cognitive competency identification using RF-CNN algorithm

Decoding Fatigue Levels of Pilots Using EEG Signals with Hybrid Deep Neural Networks

Mental workload recognition using ECG and machine learning in simulated flight tasks

Modeling and analysis of fatigue detection with multi-channel data fusion

A Pilot Workload Evaluation Method Based on EEG Data and Physiological Data

Pilots’ Fatigue Status Recognition Using Deep Contractive Autoencoder Network

EEG-Based Mental Fatigue Detection Using CNN-LSTM

Pilot Maneuvering Performance Analysis and Evaluation with Deep Learning

Noncontact perception for assessing pilot mental workload during the approach and landing under various weather conditions

LGNet: Learning Local–global EEG Representations for Cognitive Workload Classification in Simulated Flights

Research on Emotion Recognition Method of Flight Training Based on Multimodal Fusion

EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN

Classification of Drowsiness Levels Based on a Deep Spatio-Temporal Convolutional Bidirectional LSTM Network Using Electroencephalography Signals

Pilot's Cognitive State Recognition Using Wavelet Singular Entropy and Gaussian Process Classification Via Full Flight Simulation

Pilot Stress Detection Through Physiological Signals Using a Transformer-Based Deep Learning Model

Detecting Fatigue Status of Pilots Based on Deep Learning Network Using EEG Signals

Decoding EEG-based Workload Levels Using Spatio-temporal Features Under Flight Environment

Mental Pressure Recognition Method Based on CNN Model and EEG Signal under Cross Session

Cognitive State Recognition Using Wavelet Singular Entropy and ARMA Entropy with AFPA Optimized GP Classification.

An Early Warning Approach for Pilots’ Cognitive Tipping Points Based Multi-Modal Signals