Improving The Classification Accuracy of Negative Emotions Based on Multiple Physiological Signals Using A Novel Information Fusion Method FWBP

Jianhang Liang,Shuang Liu,Minghao Du,Feifan Yan,Dong Ming

DOI: https://doi.org/10.1145/3502871.3502895

2021-01-01

Abstract:Emotion recognition is crucial in the field of medical health and human-computer interaction. In recent years, using electroencephalogram (EEG) signals to identify emotions has received extensive attention and gets relatively satisfying results, while it is still a problem that the ability to identify negative emotions was limited which plays an important role in the practical application. In addition, negative emotions were rarely taken out as a major emotional state to study. Therefore, in this paper, 4 peripheral nervous system signals were collected to assist the EEG signals to improve the negative emotion recognition rate and, unlike the previous studies based on one information fusion method alone, proposed a novel information fusion method, termed as FWBP, which consists of a weighted decision fusion, a feature fusion and a BP network to achieve the information fusion of 5 physiological signals. To do this, the positive, neutral and negative emotions of 12 subjects were evoked by emotional videos, and EEG, electrodermal activity (EDA), electrocardiogram (ECG), photoplethysmogram (PPG) and respiration (RSP) were recorded simultaneously. We constructed a information fusion method FWBP. This method improved the mean accuracy by about 6% compared to central nervous system model (CNSM) which used EEG signals alone. More importantly, compared with the CNSM, the negative emotions recognition rate in this method was increased by about 10%. Furthermore, this method also proved that the identification results from multiple information fusion methods are input into BP neural network for the second fusion and classification is able to achieve complementary advantages and exhibit a better performance in emotion recognition.