Abstract:Abstract Objective. The data scarcity problem in emotion recognition from electroencephalography (EEG) leads to difficulty in building an affective model with high accuracy using machine learning algorithms or deep neural networks. Inspired by emerging deep generative models, we propose three methods for augmenting EEG training data to enhance the performance of emotion recognition models. Approach. Our proposed methods are based on two deep generative models, variational autoencoder (VAE) and generative adversarial network (GAN), and two data augmentation ways, full and partial usage strategies. For the full usage strategy, all of the generated data are augmented to the training dataset without judging the quality of the generated data, while for the partial usage, only high-quality data are selected and appended to the training dataset. These three methods are called conditional Wasserstein GAN (cWGAN), selective VAE (sVAE), and selective WGAN (sWGAN). Main results. To evaluate the effectiveness of these proposed methods, we perform a systematic experimental study on two public EEG datasets for emotion recognition, namely, SEED and DEAP. We first generate realistic-like EEG training data in two forms: power spectral density and differential entropy. Then, we augment the original training datasets with a different number of generated realistic-like EEG data. Finally, we train support vector machines and deep neural networks with shortcut layers to build affective models using the original and augmented training datasets. The experimental results demonstrate that our proposed data augmentation methods based on generative models outperform the existing data augmentation approaches such as conditional VAE, Gaussian noise, and rotational data augmentation. We also observe that the number of generated data should be less than 10 times of the original training dataset to achieve the best performance. Significance. The augmented training datasets produced by our proposed sWGAN method significantly enhance the performance of EEG-based emotion recognition models.

Enhancing facial expression recognition through generative adversarial networks-based augmentation

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Data Augmentation Using Conditional GANs for Facial Emotion Recognition

Data Augmentation and Transfer Learning Approaches Applied to Facial Expressions Recognition

Data Augmentation Techniques and Transfer Learning Approaches Applied to Facial Expressions Recognition Systems

Deep Neural Network Augmentation: Generating Faces for Affect Analysis

Evaluation of Data Augmentation Techniques for Facial Expression Recognition Systems

Face Emotion Recognization Using Dataset Augmentation Based on Neural Network

Data Augmentation in Emotion Classification Using Generative Adversarial Networks

Multi-Pose Facial Expression Recognition Based on Generative Adversarial Network

FA-GAN: Face Augmentation GAN for Deformation-Invariant Face Recognition

Compound facial expressions recognition approach using DCGAN and CNN

Joint Expression Synthesis and Representation Learning for Facial Expression Recognition

ExprGAN: Facial Expression Editing With Controllable Expression Intensity

Data Augmentation for Audio-Visual Emotion Recognition with an Efficient Multimodal Conditional GAN

Improving Speech Emotion Recognition With Adversarial Data Augmentation Network

MixAugment & Mixup: Augmentation Methods for Facial Expression Recognition

Data augmentation for enhancing EEG-based emotion recognition with deep generative models

Joint Pose and Expression Modeling for Facial Expression Recognition

Facial Expression Recognition Using Human to Animated-Character Expression Translation