Emotional Attention Detection and Correlation Exploration for Image Emotion Distribution Learning.

Zhiwei Xu,Shangfei Wang

DOI: https://doi.org/10.1109/taffc.2021.3071131

IF: 13.99

2021-01-01

IEEE Transactions on Affective Computing

Abstract:Current works on image emotion distribution learning typically extract visual representations from the holistic image or explore emotion-related regions in the image from a global-wise perspective. However, different regions of an image contribute differently to the arousal of each emotion. Existing works do not deeply explore corresponding emotion-aware regions of each emotion in the image, nor do they fully capture the relationship between each emotion-aware region and the emotion labels. In this article, we propose a novel attention based emotion distribution learning method, which can explore the emotion-related regions of images from the perspective of each emotion category, and can conduct region relationship learning. Specifically, we introduce a semantic guided attention detection network to generate class-wise attention maps for each emotion and a global-wise attention map for the holistic image. Meanwhile, an emotional graph-based network is adopted to capture the correlation between each region and the emotion distribution. Experiments on several benchmark datasets demonstrate the superiority of the proposed method compared to related works.