Noncontact perception for assessing pilot mental workload during the approach and landing under various weather conditions
Shihan Luo,Chenyang Zhang,Wenbing Zhu,Hua Chen,Jiajun Yuan,Qinyang Li,Tong Wang,Chaozhe Jiang
DOI: https://doi.org/10.1007/s11760-024-03619-x
IF: 1.583
2024-12-10
Signal Image and Video Processing
Abstract:Flight safety remains a paramount concern in the aviation industry, with pilots' mental workload playing a significant role, particularly during the approach and landing phase under various weather conditions. Monitoring pilots' mental workload in a real-time and noncontact approach is essential for preventing errors and improving flight safety. This study employs noncontact facial expression analysis using advanced sensor technology, specifically FaceReader and GoPro camera sensors, to classify pilots' mental workload during simulated approach and landing in diverse weather scenarios. Data were collected from 21 pilot cadets using a Cessna 172 simulator under three specific conditions: daytime with crosswind, nighttime with crosswind, and daytime without crosswind. The NASA-TLX scale was employed to assess subjective mental workloads. Deep learning and machine learning models including Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), K-nearest Neighbors (KNN), Decision Tree (DT), Quadratic Discriminant Analysis (QDA), Linear Discriminant Analysis (LDA), and Naive Bayes (NB) were trained on the FaceReader data. The CNN model achieved the highest performance, with an accuracy of 99.87%. SHAP (SHapley Additive exPlanations) values were utilized to interpret the model outputs, identifying 'Fearful', ' Lips Part ', and ' Chin Raiser ' as critical features. This study demonstrates the efficacy of noncontact data collection using camera sensor technology and advanced machine learning techniques in classifying mental workloads, which will enhance flight safety and improve pilot performance. Future research will focus on validating these findings in real-world scenarios and exploring the temporal evolution of mental workloads.
engineering, electrical & electronic,imaging science & photographic technology