Abstract:Affective brain-computer interfaces (aBCIs) are increasingly recognized for their potential in monitoring and interpreting emotional states through electroencephalography (EEG) signals. Current EEG-based emotion recognition methods perform well with short segments of EEG data. However, these methods encounter significant challenges in real-life scenarios where emotional states evolve over extended periods. To address this issue, we propose a Dual Attentive (DuA) transformer framework for long-term continuous EEG emotion analysis. Unlike segment-based approaches, the DuA transformer processes an entire EEG trial as a whole, identifying emotions at the trial level, referred to as trial-based emotion analysis. This framework is designed to adapt to varying signal lengths, providing a substantial advantage over traditional methods. The DuA transformer incorporates three key modules: the spatial-spectral network module, the temporal network module, and the transfer learning module. The spatial-spectral network module simultaneously captures spatial and spectral information from EEG signals, while the temporal network module detects temporal dependencies within long-term EEG data. The transfer learning module enhances the model's adaptability across different subjects and conditions. We extensively evaluate the DuA transformer using a self-constructed long-term EEG emotion database, along with two benchmark EEG emotion databases. On the basis of the trial-based leave-one-subject-out cross-subject cross-validation protocol, our experimental results demonstrate that the proposed DuA transformer significantly outperforms existing methods in long-term continuous EEG emotion analysis, with an average enhancement of 5.28%.

A transformer-based approach to video frame-level prediction in Affective Behaviour Analysis In-the-wild

Spatial-temporal Transformer for Affective Behavior Analysis

Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation

Emotion Recognition Using Transformers with Masked Learning

Multi-Task Transformer with uncertainty modelling for Face Based Affective Computing

Transformer-based Multimodal Information Fusion for Facial Expression Analysis

Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers

An Effective Ensemble Learning Framework for Affective Behaviour Analysis

DuA: Dual Attentive Transformer in Long-Term Continuous EEG Emotion Analysis

Affective Behaviour Analysis via Integrating Multi-Modal Knowledge

Affective Expression Analysis in-the-wild using Multi-Task Temporal Statistical Deep Learning Model

Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challenge

Emotion Transformer Fusion - Complementary Representation Properties of EEG and Eye Movements on Recognizing Anger and Surprise.

EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition

Frame-Transformer Emotion Classification Network.

A Simple and Interactive Transformer for Fine-Grained Emotion Detection

A Multi-term and Multi-task Analyzing Framework for Affective Analysis in-the-wild

Transformer-Based Domain Adaptation for Multi-Modal Emotion Recognition in Response to Game Animation Videos.

Multi-scale Transformer-based Network for Emotion Recognition from Multi Physiological Signals

Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos

A Unified Transformer-based Network for multimodal Emotion Recognition