DAST: A domain-adaptive learning combining spatio-temporal dynamic attention for electroencephalography emotion recognition

Hao Jin,Ying Gao,Tingting Wang,Ping Gao
DOI: https://doi.org/10.1109/jbhi.2023.3307606
IF: 7.7
2023-01-01
IEEE Journal of Biomedical and Health Informatics
Abstract:Multimodal emotion recognition with EEG-based have become mainstream in affective computing. However, previous studies mainly focus on perceived emotions (including posture, speech or face expression et.al) of different subjects, while the lack of research on induced emotions (including video or music et.al) limited the development of two-ways emotions. To solve this problem, we propose a multimodal domain adaptive method based on EEG and music called the DAST, which uses spatio-temporal adaptive attention (STA-attention) to globally model the EEG and maps all embeddings dynamically into high-dimensionally space by adaptive space encoder (ASE). Then, adversarial training is performed with domain discriminator and ASE to learn invariant emotion representations. Furthermore, we conduct extensive experiments on the DEAP dataset, and the results show that our method can further explore the relationship between induced and perceived emotions, and provide a reliable reference for exploring the potential correlation between EEG and music stimulation.
computer science, interdisciplinary applications,mathematical & computational biology,medical informatics, information systems
What problem does this paper attempt to address?