Abstract:Flight safety remains a paramount concern in the aviation industry, with pilots' mental workload playing a significant role, particularly during the approach and landing phase under various weather conditions. Monitoring pilots' mental workload in a real-time and noncontact approach is essential for preventing errors and improving flight safety. This study employs noncontact facial expression analysis using advanced sensor technology, specifically FaceReader and GoPro camera sensors, to classify pilots' mental workload during simulated approach and landing in diverse weather scenarios. Data were collected from 21 pilot cadets using a Cessna 172 simulator under three specific conditions: daytime with crosswind, nighttime with crosswind, and daytime without crosswind. The NASA-TLX scale was employed to assess subjective mental workloads. Deep learning and machine learning models including Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), K-nearest Neighbors (KNN), Decision Tree (DT), Quadratic Discriminant Analysis (QDA), Linear Discriminant Analysis (LDA), and Naive Bayes (NB) were trained on the FaceReader data. The CNN model achieved the highest performance, with an accuracy of 99.87%. SHAP (SHapley Additive exPlanations) values were utilized to interpret the model outputs, identifying 'Fearful', ' Lips Part ', and ' Chin Raiser ' as critical features. This study demonstrates the efficacy of noncontact data collection using camera sensor technology and advanced machine learning techniques in classifying mental workloads, which will enhance flight safety and improve pilot performance. Future research will focus on validating these findings in real-world scenarios and exploring the temporal evolution of mental workloads.

Pilot Intention Recognition under Risk Environment Based on Eye-movement Data

Ergonomics Analysis Based on Intention Inference

Dynamic Driving Intention Recognition of Vehicles with Different Driving Styles of Surrounding Vehicles

Assessment of Pilots’ Cognitive Competency Using Situation Awareness Recognition Model Based on Visual Characteristics

Research on Human-Computer Interaction Intention Recognition Based on EEG and Eye Movement

Research on Emotion Recognition Method of Flight Training Based on Multimodal Fusion

Decoding pilot behavior consciousness of EEG, ECG, eye movements via an SVM machine learning model

A Pilot Workload Evaluation Method Based on EEG Data and Physiological Data

Eye Movements of Pilot Flying in Simulated Flight: an Eye-Tracking Study

Mental workload recognition using ECG and machine learning in simulated flight tasks

Driver’s Intention Identification With the Involvement of Emotional Factors in Two-Lane Roads

Gaze-based Human Intention Prediction in the Hybrid Foraging Search Task

Situation Awareness Discrimination Based on Physiological Features for High-Stress Flight Tasks

Pilot's Cognitive State Recognition Using Wavelet Singular Entropy and Gaussian Process Classification Via Full Flight Simulation

Application Layer Operation Intention Recognition Based on Mouse Behavior

Modeling and analysis of fatigue detection with multi-channel data fusion

Noncontact perception for assessing pilot mental workload during the approach and landing under various weather conditions

Recognition of Different Turning Behaviors of Pilots Based on Flight Simulator and fNIRS Data

Real-Time Eye-Gaze Based Interaction for Human Intention Prediction and Emotion Analysis.

Intelligent Recognition Method of Target Tactical Behavior Intention in Air Combat Based on Deep Learning

Driver Emotion Recognition Involving Multimodal Signals: Electrophysiological Response, Nasal-Tip Temperature, and Vehicle Behavior