Abstract:Visual fatigue evaluation plays an important role in applications such as virtual reality since the visual fatigue symptoms always affect the user experience seriously. Existing visual evaluation methods require hand-crafted features for classification, and conduct feature extraction and classification in a separated manner. In this paper, we conduct a designed experiment to collect electroencephalogram (EEG) signals of various visual fatigue levels, and present a multi-scale convolutional neural network (CNN) architecture named MorletInceptionNet to detect visual fatigue using EEG as input, which exploits the spatial-temporal structure of multichannel EEG signals. Our MorletInceptionNet adopts a joint space-time-frequency features extraction scheme in which Morlet wavelet-like kernels are used for time-frequency raw feature extraction and inception architecture are further used to extract multi-scale temporal features. Then, the multi-scale temporal features are concatenated and fed to the fully connected layer for visual fatigue evaluation using classification. In experiment evaluation, we compare our method with five state-of-the-art methods, and the results demonstrate that our model achieve overally the best performance better performance for two widely used evaluation metrics, i.e., classification accuracy and kappa value. Furthermore, we use input-perturbation network-prediction correlation maps to conduct in-depth analysis into the reason why the proposed method outperforms other methods. The results suggest that our model is sensitive to the perturbation of β (14–30 Hz) and γ (30–40 Hz) bands. Furthermore, their spatial patterns are of high correlation with that of the corresponding power spectral densities which are used as evaluation features traditionally. This finding provides evidence of the hypothesis that the proposed model can learn the joint time-frequency-space features to distinguish fatigue levels automatically.

Deep Multidilation Temporal and Spatial Dependence Modeling in Stereoscopic 3-D EEG for Visual Discomfort Assessment

Learning Stereoscopic Visual Attention Model for 3d Video

3D Visual Discomfort Assessment With a Weakly Supervised Graph Convolution Neural Network Based on Inaccurately Labeled EEG

Assessment of 3D Visual Discomfort Based on Dynamic Functional Connectivity Analysis with HMM in EEG

PerimetryNet: A Multiscale Fine Grained Deep Network for Three-Dimensional Eye Gaze Estimation Using Visual Field Analysis

Monocular and Binocular Interactions Oriented Deformable Convolutional Networks for Blind Quality Assessment of Stereoscopic Omnidirectional Images

EEG-Based 3D Visual Fatigue Evaluation Using CNN

Analysis and classification of hybrid EEG features based on the depth DRDS videos

A Deep Multiscale Spatiotemporal Network for Assessing Depression from Facial Dynamics

Real-Time Non-Invasive Imaging and Detection of Spreading Depolarizations through EEG: An Ultra-Light Explainable Deep Learning Approach

3d Convolutional Long-Short Term Memory Network For Spatiotemporal Modeling Of Fmri Data

A 3D Visual Comfort Metric Based on Binocular Asymmetry Factor.

Spatiotemporal 3-D Variations Modeling with Self-Attention for Multilabel ECG Classification

Spatial-temporal 3D dependency matching with self-supervised deep learning for monocular visual sensing

OD-MVSNet: Omni-dimensional dynamic multi-view stereo network

The Abnormal Detection of Electroencephalogram With Three-Dimensional Deep Convolutional Neural Networks

Interpersonal Relationship Analysis with Dyadic EEG Signals via Learning Spatial-Temporal Patterns

Spatio-Temporal 3-Dimentional Variations Modeling with Self-Attention for Multi-Label ECG Classification

A Dual-Branch Spatio-Temporal-Spectral Transformer Feature Fusion Network for EEG-Based Visual Recognition

Deep Semantic Architecture with discriminative feature visualization for neuroimage analysis

DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery