Multi-level graph neural network for text sentiment analysis

Wenxiong Liao,Bi Zeng,Jianqi Liu,Pengfei Wei,Xiaochun Cheng,Weiwen Zhang
DOI: https://doi.org/10.1016/j.compeleceng.2021.107096
2021-06-01
Abstract:<p>Text sentiment analysis is a fundamental task in the field of natural language processing (NLP). Recently, graph neural networks (GNNs) have achieved excellent performance in various NLP tasks. However, a GNN only considers the adjacent words when updating the node representations of the graph, and thus the model can only focus on the local features while ignoring global features. In this paper, we propose a novel multi-level graph neural network (MLGNN) for text sentiment analysis. To consider both local features and global features, we apply node connection windows with different sizes at different levels. Particularly, we integrate a scaled dot-product attention mechanism as a message passing mechanism into our method for fusing the features of each word node in the graph. The experimental results demonstrated that the proposed model outperformed other models in text sentiment analysis tasks.</p>
engineering, electrical & electronic,computer science, interdisciplinary applications, hardware & architecture
What problem does this paper attempt to address?
The paper primarily addresses the fundamental task of text sentiment analysis in the field of natural language processing (NLP) and proposes a new Multi-Level Graph Neural Network (MLGNN) method. The paper points out that traditional sentiment analysis methods, such as dictionary-based methods, shallow learning methods, and deep learning methods, each have their limitations. For example, dictionary-based methods require a large amount of manual annotation and are difficult to adapt to cross-domain research; shallow learning methods are limited by manual feature engineering and have limited model expression capabilities; and existing deep learning methods, such as Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN), although they improve performance to some extent, still have some issues, such as the complex structure of RNNs making it difficult to fully utilize GPU parallelism, and CNNs only being able to capture local information. To address the above issues, the paper proposes MLGNN. This method leverages the advantages of Graph Neural Networks (GNN) and considers both local and global features through different levels of node connection windows. Specifically, MLGNN includes three levels: 1. **Bottom Level**: Uses a smaller window size to connect word nodes, focusing on local features. 2. **Intermediate Level**: Uses a larger window size, focusing on long-distance features. 3. **Top Level**: Directly connects all word nodes in the document, focusing on global features. Additionally, the paper introduces a scaled dot-product attention mechanism to integrate the features of each word node in the graph and employs a Graph Attention Network (GAT) as part of the message-passing mechanism. Experimental results show that MLGNN outperforms several commonly used models on multiple public datasets, demonstrating the effectiveness and superiority of the method.