Abstract:Frequency-domain (FD) features reveal the activated patterns of individual local brain regions responding to different emotions, whereas brain connectivity (BC) features involve the coordination of multiple brain regions for generating emotional responses; these two types of features are complementary to each other. To date, the fusion of these two types of features for electroencephalography (EEG)-based cross-subject emotion recognition remains to be fully investigated due to the intersubject variability in EEG signals. In this article, we first attempt to investigate these fused features for cross-subject emotion recognition from multiple perspectives, including critical frequency bands, complementary characteristics for each emotional state, critical channels, and crucial connections, using a fast and robust approximate empirical kernel map-fusion-based support vector machine (AEKM-Fusion-SVM) method. The experimental results on the SJTU emotion EEG dataset (SEED), BCI2020-A, and BCI2020-B datasets reveal that: 1) the AEKM-fusion method improves the effectiveness and efficiency of the fusion of features of different dimensions; 2) the recognition accuracy of the fused features outperforms each individual feature, and this outperformance is more significant in the high-frequency bands (i.e., the beta and gamma bands); 3) the fused features significantly enhance the classification performance for negative emotion; and 4) the fused features built with 27 selected channels achieve comparable performance to that of the fused features built with the full number of channels (i.e., 62 channels), allowing for easier establishment of brain–computer interface (BCI) systems in real-world scenarios. Our study enriches the research of emotion-related brain mechanisms and provides new insight into affective computing.

Research on feature fusion for emotion recognition based on discriminative canonical correlation analysis

Multi-view Domain-Adaptive Representation Learning for EEG-based Emotion Recognition

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Emotion Recognition From Multimodal Physiological Signals via Discriminative Correlation Fusion With a Temporal Alignment Mechanism

A Study of Emotion Recognition Methods Incorporating Functional Brain Network Features and Self-Attention Mechanisms

Feature-level fusion of multimodal physiological signals for emotion recognition

Multi-view Emotion Recognition Using Deep Canonical Correlation Analysis

Feature Fusion Based on Mutual-Cross-Attention Mechanism for EEG Emotion Recognition

Electroencephalogram Emotion Recognition Based on Empirical Mode Decomposition and Optimal Feature Selection.

Multimodal Emotion Recognition Using Deep Canonical Correlation Analysis

An Improved Canonical Correlation Analysis for EEG Inter-Band Correlation Extraction.

An Improved Canonical Correlation Analysis for EEG Inter-Band Correlation Extraction

Exploiting EEG signals and audiovisual feature fusion for video emotion recognition

EEG-based Emotion Recognition with Feature Fusion Networks.

Fusing Frequency-Domain Features and Brain Connectivity Features for Cross-Subject Emotion Recognition

Multidimensional Feature in Emotion Recognition Based on Multi-Channel EEG Signals.

Improving EEG-based Emotion Recognition by Fusing Time-frequency And Spatial Representations

EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM

EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention

A novel feature fusion network for multimodal emotion recognition from EEG and eye movement signals

A Novel DE-CNN-BiLSTM Multi-Fusion Model for EEG Emotion Recognition