Abstract:Abstract Background Mental workload is a critical consideration in complex man–machine systems design. Among various mental workload detection techniques, multimodal detection techniques integrating electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) signals have attracted considerable attention. However, existing EEG–fNIRS-based mental workload detection methods have certain defects, such as complex signal acquisition channels and low detection accuracy, which restrict their practical application. Methods The signal acquisition configuration was optimized by analyzing the feature importance in mental workload recognition model and a more accurate and convenient EEG–fNIRS-based mental workload detection method was constructed. A classical Multi-Task Attribute Battery (MATB) task was conducted with 20 participating volunteers. Subjective scale data, 64-channel EEG data, and two-channel fNIRS data were collected. Results A higher number of EEG channels correspond to higher detection accuracy. However, there is no obvious improvement in accuracy once the number of EEG channels reaches 26, with a four-level mental workload detection accuracy of 76.25 ± 5.21%. Partial results of physiological analysis verify the results of previous studies, such as that the θ power of EEG and concentration of O 2 Hb in the prefrontal region increase while the concentration of HHb decreases with task difficulty. It was further observed, for the first time, that the energy of each band of EEG signals was significantly different in the occipital lobe region, and the power of $$\beta_{1}$$ β 1 and $$\beta_{2}$$ β 2 bands in the occipital region increased significantly with task difficulty. The changing range and the mean amplitude of O 2 Hb in high-difficulty tasks were significantly higher compared with those in low-difficulty tasks. Conclusions The channel configuration of EEG–fNIRS-based mental workload detection was optimized to 26 EEG channels and two frontal fNIRS channels. A four-level mental workload detection accuracy of 76.25 ± 5.21% was obtained, which is higher than previously reported results. The proposed configuration can promote the application of mental workload detection technology in military, driving, and other complex human–computer interaction systems.

Synthesized Quantitative Assessment of Human Mental Fatigue with EEG and HRV

Quantitative analysis of heart rate variability parameter and mental stress index

Assessment of Mental Workload Based on Multi-Physiological Signals

Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces

Heart Rate Variability-Based Subjective Physical Fatigue Assessment

A Practical Application for Quantitative Brain Fatigue Evaluation Based on Machine Learning and Ballistocardiogram

Objective Assessments of Mental Fatigue During a Continuous Long-Term Stress Condition

The impact of mental fatigue on brain activity: a comparative study both in resting state and task state using EEG

Optimized electroencephalogram and functional near-infrared spectroscopy-based mental workload detection method for practical applications

Mental workload recognition using ECG and machine learning in simulated flight tasks

Quantitative identification of daily mental fatigue levels based on multimodal parameters

Towards Automated Fatigue Assessment using Wearable Sensing and Mixed-Effects Models

Spatiotemporal Dynamical Analysis of Brain Activity During Mental Fatigue Process

Intelligent fatigue detection based on hierarchical multi-scale ECG representations and HRV measures

Comparison Analysis of Different Time-Scale Heart Rate Variability Signals for Mental Workload Assessment in Human-Robot Interaction

Study on the nonfatigue and fatigue states of orchard workers based on electrocardiogram signal analysis

EEG-Based Mental Workload Assessment in a Simulated Flight Task Using Single-Channel Brain Rhythm Sequence

Pre-service Fatigue Screening for Construction Workers Through Wearable EEG-based Signal Spectral Analysis

Functional Connectivity Analysis and Detection of Mental Fatigue Induced by Different Tasks Using Functional Near-Infrared Spectroscopy

Data-driven natural computational psychophysiology in class

Application of quantitative EEG analysis to workload assessment in an advanced aircraft simulator