Abstract:Emotion constitutes a higher-order cognitive process, moreover, physiological signals can offer amore objective and realistically reflection of human emotional state. Electroencephalograph (EEG) signals, controlled by the central nervous system, are highly sensitive to emotional state fluctuations. Current research primarily focuses on recognizing emotions through feature extraction from multi-channel EEG signals. However, the acquisition of multi-channel EEG signals is hindered by difficulties acquiring from hair occlusion. The objective of this study is to construct a real-time physiological-emotion recognition model based on 2-channel EEG signals from the forehead without hair occlusion combined with 3-channel peripheral physiological (PPS) signals, employing signal preprocessing, feature extraction, feature normalization, and the ensemble learning algorithm Light Gradient Boosting Machine (LightGBM). In this paper, thirty participants sequentially viewed fourteen virtual reality (VR) scenes, rated for valence and arousal using the Self-Assessment Manikin (SAM) scale after each scene. The 2-channel EEG signals and 3-channel PPS signals were recorded synchronously from forehead by VR Head-Mounted Displays (HMD) Bio Pad developed in our research center. Our proposed model achieved median accuracy of 89.68% for valence, 90.11% for arousal in binary classification, and 84.93% for four classification of emotion recognition based on the collected samples. Furthermore, verification result of the proposed method on a public database (DEAP) achieved accuracy of 84.03%, 84.37%, and 72.23% on valence, arousal, and four classification, respectively. The proposed physiological-emotion recognition model, utilizing a limited number of multimodal physiological signals channels from wearable devices, demonstrates higher accuracy compared with the results in some literature, suggesting its potential for real-time implementation due to limited signal channels and low runtime.

Situation Awareness Recognition Using EEG and Eye-Tracking Data: a Pilot Study.

Physiological Records-Based Situation Awareness Evaluation under Aviation Context: A Comparative Analysis

Sustained Attention States Recognition with EEG and Eye-Tracking in the GradCPT.

EEG-based Recognition of Driver State Related to Situation Awareness Using Graph Convolutional Networks

Recognizing situation awareness of forklift operators based on eye-movement & EEG features

Situation Awareness Discrimination Based on Physiological Features for High-Stress Flight Tasks

-wave-based exploration of sensitive EEG features and classification of situation awareness

Mobile EEG-based situation awareness recognition for air traffic controllers

Physiological Measurements of Situation Awareness: A Systematic Review

EEG Feature Analysis Related to Situation Awareness Assessment and Discrimination

Recognizing the situation awareness of forklift operators based on EEG techniques in a field experiment

Subject Matching for Cross-Subject EEG-based Recognition of Driver States Related to Situation Awareness.

Multi-modal emotion analysis from facial expressions and electroencephalogram.

Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

An Eye-Fixation Related Electroencephalography Technique for Predicting Situation Awareness: Implications for Driver State Monitoring Systems

Multimodal Affective State Assessment Using fNIRS + EEG and Spontaneous Facial Expression

Use of multimodal physiological signals to explore pilots’ cognitive behaviour during flight strike task performance

Multimodal Emotion Recognition by Combining Physiological Signals and Facial Expressions: a Preliminary Study.

Multimodal Sensing and Computational Intelligence for Situation Awareness Classification in Autonomous Driving

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

Emotion Recognition Based on a Limited Number of Multimodal Physiological Signals Channels