Abstract:Mental workload is defined as the proportion of the information processing capability used to perform a task. High cognitive load requires additional resources to process information; this demand for additional resources may reduce the processing efficiency and performance. Therefore, the technique of workload estimation can ensure a proper working environment to promote the working efficiency of each person. In this paper, we propose a three-dimensional convolutional neural network (3D CNN) employing a multilevel feature fusion algorithm for mental workload estimation using electroencephalogram (EEG) signals. The 1D EEG signals are converted to 3D EEG images to enable the 3D CNN to learn the spectral and spatial information over the scalp. The multilevel feature fusion framework integrates local and global neuronal activities by workload tasks in the 3D CNN algorithm. Multilevel features are extracted in each layer of the 3D convolution operation and each multilevel feature is multiplied by a weighting factor, which determines the importance of the feature. The weighting factor is adaptively estimated for each EEG image by a backpropagation process. Furthermore, we generate subframes from each EEG image and propose a temporal attention technique based on the long short-term memory model (LSTM) to extract a significant subframe at each multilevel feature that is strongly correlated with task difficulty. To verify the performance of our network, we performed the Sternberg task to measure the mental workload of the participant, which was classified according to its difficulty as low or high workload condition. We showed that the difficulty of the workload was well designed, which was reflected in the behavior of the participant. Our network is trained on this dataset and the accuracy of our network is 90.8 %, which is better than that of conventional algorithms. We also evaluated our method using the public EEG dataset and achieved 93.9 % accuracy.

Identifying stable EEG patterns over time for mental workload recognition using transfer DS-CNN framework

Assessing Cognitive Mental Workload Via EEG Signals and an Ensemble Deep Learning Classifier Based on Denoising Autoencoders

Topological EEG-Based Functional Connectivity Analysis for Mental Workload State Recognition

Pattern Recognition of Momentary Mental Workload Based on Multi-Channel Electrophysiological Data and Ensemble Convolutional Neural Networks

Identification of Mental Workload Using Imbalanced EEG Data and DySMOTE-based Neural Network Approach

Identification of Temporal Variations in Mental Workload Using Locally-Linear-embedding-based EEG Feature Reduction and Support-Vector-machine-based Clustering and Classification Techniques

Identifying mental fatigue of construction workers using EEG and deep learning

Spectral and Temporal Feature Learning with Two-Stream Neural Networks for Mental Workload Assessment

Mental workload recognition using ECG and machine learning in simulated flight tasks

Learning Spatial–Spectral–Temporal EEG Features With Recurrent 3D Convolutional Neural Networks for Cross-Task Mental Workload Assessment

WLnet: Towards an Approach for Robust Workload Estimation Based on Shallow Neural Networks

Cross-Task Mental Workload Recognition Based on EEG Tensor Representation and Transfer Learning

Mental Workload Recognition by Combining Wavelet Packet Transform and Kernel Spectral Regression Techniques

Pattern Classification of Instantaneous Mental Workload Using Ensemble of Convolutional Neural Networks

Generic Mental Workload Measurement Using a Shared Spatial Map Network With Different EEG Channel Layouts

A Mental Workload Classification Method Based on GCN Modified by Squeeze-and-Excitation Residual

Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling

Multilevel Feature Fusion With 3D Convolutional Neural Network for EEG-Based Workload Estimation

Mental fatigue recognition study based on 1D convolutional neural network and short-term ECG signals

Mental workload estimation with electroencephalogram signals by combining multi-space deep models

Deep long short-term memory structures model temporal dependencies improving cognitive workload estimation