Zebin JiangXinyan LiLiezhong GeJie XuYandi LuYijing ZhangMing Maoa Center for Psychological Sciences,Zhejiang University,Hangzhou,Chinab China North Vehicle Research Institute,Beijing,ChinaZebin Jiang is a PhD candidate at the Center for Psychological Sciences at Zhejiang University. He obtained his bachelor's degree in Computer Science and Technology from Zhengzhou University in 2021. His research interests include status recognition and human-machine interaction.Xinyan Li is a researcher fellow at the China North Vehicle Research Institute. She has long been engaged in the research of intelligent human-machine interaction-related technologies for special vehicles. She is committed to exploring methods of human-machine interaction and application modes suitable for special vehicles.Liezhong Ge is a professor at the Center for Psychological Sciences at Zhejiang University. He has been engaged in teaching and research in engineering psychology and cognitive psychology for more than three decades. His research interests include human-machine interaction,user experience and product usability,and facial recognition research.Jie Xu is an Assistant Professor at the Center for Psychological Sciences at Zhejiang University. He obtained his PhD in industrial and systems engineering from the University of Wisconsin–Madison in 2016. His research interests include human-agent interaction,physiological computing,and system safety.Yandi Lu is a PhD candidate at the Center for Psychological Sciences at Zhejiang University. He has an interdisciplinary background in design and psychology. His research interests include human–computer interaction,user experience,and human-centered AI.Yijing Zhang is currently a postdoctoral fellow at the Center for Psychological Sciences at Zhejiang University. He employs cognitive neuroscience approaches to delve into various aspects of research. His areas of focus include the investigation and regulation of mental workload,human-machine interaction,and special vehicle driver state recognition.Ming Mao is an academician of the Chinese Academy of Sciences. He has long been engaged in theoretical research and engineering practice on the overall design of armored weapons,drive systems,action systems,and other technologies.

Abstract:Mental workload recognition is of great significance in preventing human errors and accidents. This study constructed a multimodal recognition scheme to recognize three mental workload states: underload, moderate load, and overload. Based on driving scenarios, these three states were induced in this study by changing the driving modes and situations. Multimodal recognition of underload, moderate load, and overload was performed using electroencephalography (EEG), electrocardiography (ECG), and pupillometry. In addition, various machine learning methods were used to evaluate the recognition performance of different feature combinations. The results showed that the random forest method, trained using spectral power, pupil diameter, and heart rate variability, achieved the highest recognition accuracy of 83.13% for the three mental workload states. This study provides valuable reference information for multimodal recognition of mental workload states.

Mental Workload Classification Based on Semi-Supervised Extreme Learning Machine

Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling

Recognition of Mental Workload Levels under Complex Human–Machine Collaboration by Using Physiological Features and Adaptive Support Vector Machines

Mental workload recognition using ECG and machine learning in simulated flight tasks

Recognition of mental workload levels by combining adaptive exponential feature smoothing and locality preservation projection techniques

Identification of Temporal Variations in Mental Workload Using Locally-Linear-embedding-based EEG Feature Reduction and Support-Vector-machine-based Clustering and Classification Techniques

Neurophysiological Feature-Based Detection of Mental Workload by Ensemble Support Vector Machines

A Deep Learning Scheme for Mental Workload Classification Based on Restricted Boltzmann Machines

Identifying Mental Workload Using EEG and Deep Learning

Mental Workload Recognition by Combining Wavelet Packet Transform and Kernel Spectral Regression Techniques

Assessing Cognitive Mental Workload Via EEG Signals and an Ensemble Deep Learning Classifier Based on Denoising Autoencoders

Cognitive Workload Recognition Using EEG Signals and Machine Learning: A Review

EEG-Based Mental Workload Classification Method Based on Hybrid Deep Learning Model under IoT.

EEG-Based Mental Workload Assessment in a Simulated Flight Task Using Single-Channel Brain Rhythm Sequence

Physiological Measures of Mental Workload: Evidence from Empirical Studies

Mental Workload Classification Based on Ignored Auditory Probes and Spatial Covariance

Towards Mental Load Assessment for High-Risk Works Driven by Psychophysiological Data: Combining a 1D-CNN Model with Random Forest Feature Selection

Identification of Cognitive Workload during Surgical Tasks with Multimodal Deep Learning

Using Multimodal Methods and Machine Learning to Recognize Mental Workload: Distinguishing Between Underload, Moderate Load, and Overload

Nonlinear Dynamic Classification of Momentary Mental Workload Using Physiological Features and Narx-Model-Based Least-Squares Support Vector Machines

Assessment of Mental Workload Based on Multi-Physiological Signals