Abstract:Abstract Background Mental workload is a critical consideration in complex man–machine systems design. Among various mental workload detection techniques, multimodal detection techniques integrating electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals have attracted considerable attention. However, existing EEG–fNIRS-based mental workload detection methods have certain defects, such as complex signal acquisition channels and low detection accuracy, which restrict their practical application. Methods The signal acquisition configuration was optimized by analyzing the feature importance in mental workload recognition model and a more accurate and convenient EEG–fNIRS-based mental workload detection method was constructed. A classical Multi-Task Attribute Battery (MATB) task was conducted with 20 participating volunteers. Subjective scale data, 64-channel EEG data, and two-channel fNIRS data were collected. Results A higher number of EEG channels correspond to higher detection accuracy. However, there is no obvious improvement in accuracy once the number of EEG channels reaches 26, with a four-level mental workload detection accuracy of 76.25 ± 5.21%. Partial results of physiological analysis verify the results of previous studies, such as that the θ power of EEG and concentration of O 2 Hb in the prefrontal region increase while the concentration of HHb decreases with task difficulty. It was further observed, for the first time, that the energy of each band of EEG signals was significantly different in the occipital lobe region, and the power of $$\beta_{1}$$ β 1 and $$\beta_{2}$$ β 2 bands in the occipital region increased significantly with task difficulty. The changing range and the mean amplitude of O 2 Hb in high-difficulty tasks were significantly higher compared with those in low-difficulty tasks. Conclusions The channel configuration of EEG–fNIRS-based mental workload detection was optimized to 26 EEG channels and two frontal fNIRS channels. A four-level mental workload detection accuracy of 76.25 ± 5.21% was obtained, which is higher than previously reported results. The proposed configuration can promote the application of mental workload detection technology in military, driving, and other complex human–computer interaction systems.

Cross-Task Mental Workload Recognition Based on EEG Tensor Representation and Transfer Learning

Cross-Task Cognitive Workload Recognition Based on EEG and Domain Adaptation

Towards an Effective Cross-Task Mental Workload Recognition Model Using Electroencephalography Based on Feature Selection and Support Vector Machine Regression.

Mental workload recognition using ECG and machine learning in simulated flight tasks

EEG Based Dynamic Functional Connectivity Analysis in Mental Workload Tasks With Different Types of Information

An EEG-based Mental Workload Estimator Trained on Working Memory Task Can Work Well under Simulated Multi-Attribute Task.

EEG-based Mental Workload Recognition Related to Multitasking

Learning Spatial–Spectral–Temporal EEG Features With Recurrent 3D Convolutional Neural Networks for Cross-Task Mental Workload Assessment

EEG-Based Mental Workload Classification Method Based on Hybrid Deep Learning Model under IoT.

Optimized electroencephalogram and functional near-infrared spectroscopy-based mental workload detection method for practical applications

Identifying stable EEG patterns over time for mental workload recognition using transfer DS-CNN framework

EEG-TNet: an End-To-End Brain Computer Interface Framework for Mental Workload Estimation

Cross-task cognitive workload recognition using a dynamic residual network with attention mechanism based on neurophysiological signals

Cross-Subject Cognitive Workload Recognition Based on EEG and Deep Domain Adaptation

Evaluation of Mental Workload in Working Memory Tasks with Different Information Types Based on EEG.

Classification of mental workload using brain connectivity and machine learning on electroencephalogram data

Mental workload estimation with electroencephalogram signals by combining multi-space deep models

Topological EEG-Based Functional Connectivity Analysis for Mental Workload State Recognition

SCVCNet: Sliding cross-vector convolution network for cross-task and inter-individual-set EEG-based cognitive workload recognition

Spectral and Temporal Feature Learning with Two-Stream Neural Networks for Mental Workload Assessment