Abstract:Multimodal bio-signals acquisition based on wearable devices and using virtual reality (VR) as stimulus source are promising techniques in emotion recognition research field. Numerous studies have shown that emotional states can be better evoked through Immersive Virtual Environments (IVE). The main goal of this paper is to provide researchers with a system for emotion recognition in VR environments. In this paper, we present a wearable forehead bio-signals acquisition pad which is attached to Head-Mounted Displays (HMD), termed HMD Bio Pad. This system can simultaneously record emotion-related two-channel electroencephalography (EEG), one-channel electrodermal activity (EDA), photoplethysmograph (PPG) and skin temperature (SKT) signals. In addition, we develop a human-computer interaction (HCI) interface which researchers can carry out emotion recognition research using VR HMD as stimulus presentation device. To evaluate the performance of the proposed system, we conducted different experiments to validate the multimodal bio-signals quality, respectively. To validate EEG signal, we have assessed the performance in terms of EEG eyes-blink task and eyes-open and eyes-closed task. The EEG eyes-blink task indicates that the proposed system can achieve comparable EEG signal quality in comparison to the dedicated bio-signals measuring device. The eyes-open and eyes-closed task proves that the proposed system can efficiently record alpha rhythm. Then we used signal-to-noise ratio (SNR) and Skin Conductance Reaction (SCR) signal to validate the performance for EDA acquisition system. A filtered EDA signal, with a high mean SNR of 28.52 dB, is plotted on HCI interface. Moreover, the SCR signal related to stimulus response can be correctly extracted from EDA signal. The SKT acquisition system has been validated effectively by the temperature change experiment when subjects are in unpleasant emotion. The pulse rate (PR) estimated from PPG signal achieved the low mean average absolute error (AAE), which is 1.12 beats per minute (BPM) over 8 recordings. In summary, the proposed HMD Bio Pad offers a portable, comfortable and easy-to-wear device for recording bio-signals. The proposed system could contribute to emotion recognition research in VR environments.

Cross-subject And Cross-device Wearable EEG Emotion Recognition Using Frontal EEG Under Virtual Reality Scenes

Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System

A Dataset for Emotion Recognition Using Virtual Reality and EEG (DER-VREEG): Emotional State Classification Using Low-Cost Wearable VR-EEG Headsets

EEG-based cross-subject emotion recognition using multi-source domain transfer learning

A Wearable Head Mounted Display Bio-Signals Pad System for Emotion Recognition

Electroencephalography-based recognition of six basic emotions in virtual reality environments

EEG-based emotion recognition in an immersive virtual reality environment: From local activity to brain network features

Explorations of A Real-Time VR Emotion Prediction System Using Wearable Brain-Computer Interfacing

Multi-source joint domain adaptation for cross-subject and cross-session emotion recognition from electroencephalography

Multi-Source EEG Emotion Recognition via Dynamic Contrastive Domain Adaptation

Emotion Recognition in HMDs: A Multi-task Approach Using Physiological Signals and Occluded Faces

Human Emotions Analysis and Recognition Using EEG Signals in Response to 360$^\circ$ Videos

Multi-Source and Multi-Representation Adaptation for Cross-Domain Electroencephalography Emotion Recognition

Wearable Wireless Dual-Channel EEG System for Emotion Recognition Based on Machine Learning

Multi-subject Subspace Alignment for Non-Stationary EEG-based Emotion Recognition

Plug-and-Play Domain Adaptation for Cross-Subject EEG-based Emotion Recognition

Domain-adaptive emotion recognition based on horizontal vertical flow representation of EEG signals

An Efficient Machine Learning-Based Emotional Valence Recognition Approach Towards Wearable EEG

[Research progress on emotion recognition by combining virtual reality environment and electroencephalogram signals]

Multi-source Selective Graph Domain Adaptation Network for cross-subject EEG emotion recognition

Naturalistic Emotion Recognition Using EEG and Eye Movements