Co-attention networks based on aspect and context for aspect-level sentiment analysis

MeiZhen Liu,FengYu Zhou,Ke Chen,Yang Zhao
DOI: https://doi.org/10.1016/j.knosys.2021.106810
2021-04-01
Abstract:<p>Aspect-level sentiment analysis aims to predict the sentiment orientation of a specific aspect in one context. Recent studies have achieved great success in modeling aspect and context by applying long-short term memory networks (LSTMs) and an attention mechanism. However, the semantic correlations between each word of the aspect and context fail to be noticed, which decreases the effectiveness of feature representations. Given this problem, in order to comprehensively analyze semantic correlations from the perspective of the word level and feature level, a co-attention mechanism is proposed to capture the interactions between aspect and context, which interactively concentrates the semantic influences on context and aspect to generate greater informative representation. Specifically, a co-attention mechanism consists of the 1-pair hop mechanism and an interactive mechanism, in which the 1-pair hop mechanism pays more attention to the important word's aspect or context and the interactive mechanism highlights the significant feature of the aspect or context by calculating the interactive attention matrix from the perspective of the feature level. In addition, considering that one context contains more than one aspect, the novel loss function is designed to fully employ the attention weights of different aspects on every word in the given context. Extensive compared experiments are conducted based on the GloVe and BERT pre-trained models, and the results show that the proposed method can achieve state-of-the-art performance on the Restaurant and Twitter datasets. Furthermore, ablation studies are designed to validate the necessity and importance of the 1-pair hop mechanism and interactive mechanism.</p>
computer science, artificial intelligence
What problem does this paper attempt to address?
The paper primarily focuses on addressing the problem of fine-grained sentiment analysis, specifically refining sentiment analysis at the sentence or document level to aspect-based sentiment polarity classification. In traditional sentence or document-level sentiment analysis, the sentiment polarity differences of various aspects are often overlooked. For example, in the sentence describing a restaurant, "The price is reasonable, but the service is poor," it contains two different aspects of the restaurant: "price" and "service," and the sentiment polarities of these two aspects are different. Therefore, the paper proposes a method based on a collaborative attention mechanism to solve this problem. Specifically, the paper makes the following contributions: 1. **Proposes an effective aspect-based fine-grained sentiment analysis method**: By using a collaborative attention mechanism, it addresses the semantic association between specific aspects and the context, thereby enhancing the expressiveness of text feature extraction. 2. **Designs a novel loss function**: This loss function is based on the attention distribution of specific aspects obtained through the collaborative attention mechanism, which can fully utilize the attention weight information, enabling the model to perform more accurate classification. 3. **Conducts comparative experiments and ablation studies**: Based on two pre-trained language models (GloVe and BERT), the proposed method achieves the latest best results on the SemEval 2014, 2016 restaurant datasets, and the Twitter dataset. In summary, the paper aims to propose a new collaborative attention network to comprehensively analyze the semantic association between each aspect word and the context from both the word level and the feature level, and designs a new loss function to improve the accuracy of sentiment classification.