Zebin JiangXinyan LiLiezhong GeJie XuYandi LuYijing ZhangMing Maoa Center for Psychological Sciences,Zhejiang University,Hangzhou,Chinab China North Vehicle Research Institute,Beijing,ChinaZebin Jiang is a PhD candidate at the Center for Psychological Sciences at Zhejiang University. He obtained his bachelor's degree in Computer Science and Technology from Zhengzhou University in 2021. His research interests include status recognition and human-machine interaction.Xinyan Li is a researcher fellow at the China North Vehicle Research Institute. She has long been engaged in the research of intelligent human-machine interaction-related technologies for special vehicles. She is committed to exploring methods of human-machine interaction and application modes suitable for special vehicles.Liezhong Ge is a professor at the Center for Psychological Sciences at Zhejiang University. He has been engaged in teaching and research in engineering psychology and cognitive psychology for more than three decades. His research interests include human-machine interaction,user experience and product usability,and facial recognition research.Jie Xu is an Assistant Professor at the Center for Psychological Sciences at Zhejiang University. He obtained his PhD in industrial and systems engineering from the University of Wisconsin–Madison in 2016. His research interests include human-agent interaction,physiological computing,and system safety.Yandi Lu is a PhD candidate at the Center for Psychological Sciences at Zhejiang University. He has an interdisciplinary background in design and psychology. His research interests include human–computer interaction,user experience,and human-centered AI.Yijing Zhang is currently a postdoctoral fellow at the Center for Psychological Sciences at Zhejiang University. He employs cognitive neuroscience approaches to delve into various aspects of research. His areas of focus include the investigation and regulation of mental workload,human-machine interaction,and special vehicle driver state recognition.Ming Mao is an academician of the Chinese Academy of Sciences. He has long been engaged in theoretical research and engineering practice on the overall design of armored weapons,drive systems,action systems,and other technologies.

Abstract:Mental workload recognition is of great significance in preventing human errors and accidents. This study constructed a multimodal recognition scheme to recognize three mental workload states: underload, moderate load, and overload. Based on driving scenarios, these three states were induced in this study by changing the driving modes and situations. Multimodal recognition of underload, moderate load, and overload was performed using electroencephalography (EEG), electrocardiography (ECG), and pupillometry. In addition, various machine learning methods were used to evaluate the recognition performance of different feature combinations. The results showed that the random forest method, trained using spectral power, pupil diameter, and heart rate variability, achieved the highest recognition accuracy of 83.13% for the three mental workload states. This study provides valuable reference information for multimodal recognition of mental workload states.

Recognition of driver’s mental workload based on physiological signals, a comparative study

Mental workload recognition using ECG and machine learning in simulated flight tasks

Classification of Driver Cognitive Load in Conditionally Automated Driving: Utilizing Electrocardiogram-Based Spectrogram with Lightweight Neural Network

Using Multimodal Methods and Machine Learning to Recognize Mental Workload: Distinguishing Between Underload, Moderate Load, and Overload

Physiological Indicators of Driver Workload During Car-Following Scenarios and Takeovers in Highly Automated Driving

Driver Mental Fatigue Detection Based on Head Posture Using New Modified reLU-BiLSTM Deep Neural Network

Characterisation of Cognitive Load Using Machine Learning Classifiers of Electroencephalogram Data

Classification of Drivers’ Mental Workload Levels: Comparison of Machine Learning Methods Based on ECG and Infrared Thermal Signals

Driving Drowsiness Detection With Eeg Using A Modified Hierarchical Extreme Learning Machine Algorithm With Particle Swarm Optimization: A Pilot Study

EEG classification of driver mental states by deep learning

Detection of driver fatigue driving behaviour based on deep learning algorithm from the perspective of traffic psychology

Machine Learning Based Driver Emotion Monitoring for Vehicular IoT

A LightGBM-Based EEG Analysis Method for Driver Mental States Classification

Assessment of Mental Workload Based on Multi-Physiological Signals

Assessment of Drivers' Mental Workload by Multimodal Measures during Auditory-Based Dual-Task Driving Scenarios

Driving Cognitive Alertness Detecting Using Evoked Multimodal Physiological Signals Based on Uncertain Self-Supervised Learning

Deep Learning for Detecting Multi-Level Driver Fatigue Using Physiological Signals: A Comprehensive Approach

Driving-Induced Neurological Biomarkers in an Advanced Driver-Assistance System

Investigating mental workload caused by NDRTs in highly automated driving with deep learning

Driver Profiling and Bayesian Workload Estimation Using Naturalistic Peripheral Detection Study Data

Classification of Driver Cognitive Load: Exploring the Benefits of Fusing Eye-Tracking and Physiological Measures