Abstract:Visual fatigue evaluation plays an important role in applications such as virtual reality since the visual fatigue symptoms always affect the user experience seriously. Existing visual evaluation methods require hand-crafted features for classification, and conduct feature extraction and classification in a separated manner. In this paper, we conduct a designed experiment to collect electroencephalogram (EEG) signals of various visual fatigue levels, and present a multi-scale convolutional neural network (CNN) architecture named MorletInceptionNet to detect visual fatigue using EEG as input, which exploits the spatial-temporal structure of multichannel EEG signals. Our MorletInceptionNet adopts a joint space-time-frequency features extraction scheme in which Morlet wavelet-like kernels are used for time-frequency raw feature extraction and inception architecture are further used to extract multi-scale temporal features. Then, the multi-scale temporal features are concatenated and fed to the fully connected layer for visual fatigue evaluation using classification. In experiment evaluation, we compare our method with five state-of-the-art methods, and the results demonstrate that our model achieve overally the best performance better performance for two widely used evaluation metrics, i.e., classification accuracy and kappa value. Furthermore, we use input-perturbation network-prediction correlation maps to conduct in-depth analysis into the reason why the proposed method outperforms other methods. The results suggest that our model is sensitive to the perturbation of β (14–30 Hz) and γ (30–40 Hz) bands. Furthermore, their spatial patterns are of high correlation with that of the corresponding power spectral densities which are used as evaluation features traditionally. This finding provides evidence of the hypothesis that the proposed model can learn the joint time-frequency-space features to distinguish fatigue levels automatically.

Detecting Dynamic Behavior of Brain Fatigue Through 3-D-CNN-LSTM

Spatial-Temporal Feature Representation Learning for Facial Fatigue Detection.

A Driving Performance Forecasting System Based on Brain Dynamic State Analysis Using 4-D Convolutional Neural Networks

EEG-Based Mental Fatigue Detection Using CNN-LSTM

Understanding Cognitive Fatigue from fMRI Scans with Self-supervised Learning

Linking Attention-Based Multiscale CNN With Dynamical GCN for Driving Fatigue Detection

Driving Fatigue Detection Based on the Combination of Multi-Branch 3D-CNN and Attention Mechanism

EEG-Based 3D Visual Fatigue Evaluation Using CNN

EEG-Based Spatio-Temporal Convolutional Neural Network for Driver Fatigue Evaluation

Spatiotemporal Dynamical Analysis of Brain Activity During Mental Fatigue Process

Remote Photoplethysmography and Motion Tracking Convolutional Neural Network with Bidirectional Long Short-Term Memory: Non-Invasive Fatigue Detection Method Based on Multi-Modal Fusion

A Lightweight Attention-Based Multi-Frequency Topology Learning Framework for Driving Fatigue Detection

A novel deep-learning model based on τ-shaped convolutional network (τNet) with long short-term memory (LSTM) for physiological fatigue detection from EEG and EOG signals

Machine vision detection to daily facial fatigue with a nonlocal 3D attention network

Fatigue Detection with Spatial-Temporal Fusion Method on Covariance Manifolds of Electroencephalography.

Mental fatigue recognition study based on 1D convolutional neural network and short-term ECG signals

Fatigue Driving State Detection Based on Spatial Characteristics of EEG Signals

A Recurrence Network-Based Convolutional Neural Network for Fatigue Driving Detection from EEG.

[Mental fatigue state recognition method based on convolution neural network and long short-term memory]

Attention-based multi-semantic dynamical graph convolutional network for eeg-based fatigue detection

Nonparametric Bayesian Prior Inducing Deep Network for Automatic Detection of Cognitive Status