Abstract:Traditional emotion recognition technology often focuses on recognizing human biometrics such as facial expressions or body postures. However, psychological research shows that the context (contextual information) also plays an important role in perceiving the emotions of others. Existing research methods, that are based on contextual information, have heavily relied on the semantic features of images. They do not take into account the interrelationships between objects and fail to use external knowledge. Meanwhile, external knowledge is likely to be very helpful in perceiving emotion. In this paper, by incorporating external structured emotion commonsense knowledge, two methods are proposed for constructing emotion knowledge graphs based on the objective text of images, and a multi-modal emotion recognition model is designed. The model has three branches, one of which focuses on human biometrics, and another two branches employ emotion knowledge graphs to perceive emotion from contextual information. Before constructing the emotion knowledge graphs, we convert the visual content into the text information to obtain the prime and ample contextual information from the object, scene, and the relationship between the objects. This approach can reduce redundant and invalid information. After that, the structured emotion commonsense knowledge is integrated into the objective text in word sharing. A large-scale emotion knowledge graph based on all valid words (LEKG) and a small-scale emotion knowledge graph based on the document itself (TEKG) are constructed, respectively. We propose two fusion modules, one of which is attention-based, and the other is a deep reasoning module that incorporates interpersonal relation. We conduct extensive experiments on the benchmark dataset EMOTIC. The experimental results prove that our method is superior to the most advanced methods, and it has obvious advantages in global context-aware tasks.

Chinese Text-Oriented Interpersonal Emotional Relationship Mining Based on Event Knowledge Model

Community Mining From Multi-Relational Networks

Mining Social Emotions from Affective Text

Interpretable Emotion Analysis Based on Knowledge Graph and OCC Model

An Improved Method of Extracting Emotional Attribute of Relations Based on Syntactical Analysis

Ontology-Based Model For Mining User'S Emotions On The Wisdom Web

Hot news mining and public opinion guidance analysis based on sentiment computing in network social media

Text-based emotion classification using emotion cause extraction

Prior Knowledge-Based Event Network for Chinese Text

A Study of Entity Relationship Extraction Algorithms Based on Symmetric Interaction Between Data, Models, and Inference Algorithms

An Algorithm for User Interest Modeling Using Facial Expression Recognition.

CCKS 2019 Shared Task on Inter-Personal Relationship Extraction

Emotional Community Detection in Social Network

Extracting Event Temporal Information Based on Web

Research on Emotional Interaction and User Cognition

Incorporating Structured Emotion Commonsense Knowledge and Interpersonal Relation into Context-Aware Emotion Recognition

Emotion Correlation Mining Through Deep Learning Models on Natural Language Text

Event Relationship Analysis for Temporal Event Search.

Event Extraction with Deep Contextualized Word Representation and Multi-attention Layer

Public Behavior and Emotion Correlation Mining Driven by Aspect From News Corpus