Abstract:Graph-based neural networks and unsupervised pre-trained models are both cutting-edge text representation methods, given their outstanding ability to capture global information and contextualized information, respectively. However, both representation methods meet obstacles to further performance improvements. On one hand, graph-based neural networks lack knowledge orientation to guide textual interpretation during global information interaction. On the other hand, unsupervised pre-trained models imply rich semantic and syntactic knowledge which lacks sufficient induction and expression. Therefore, how to effectively integrate graph-based global information and unsupervised contextualized semantic and syntactic information to achieve better text representation is an important issue pending for solution. In this paper, we propose a representation method that deeply integrates Unsupervised Semantics and Syntax into heterogeneous Graphs (USS-Graph) for inductive text classification. By constructing a heterogeneous graph whose edges and nodes are totally generated by knowledge from unsupervised pre-trained models, USS-Graph can harmonize the two perspectives of information under a bidirectionally weighted graph structure and thereby realizing the intra-fusion of graph-based global information and unsupervised contextualized semantic and syntactic information. Based on USS-Graph, we also propose a series of optimization measures to further improve the knowledge integration and representation performance. Extensive experiments conducted on benchmark datasets show that USS-Graph consistently achieves state-of-the-art performances on inductive text classification tasks. Additionally, extended experiments are conducted to deeply analyze the characteristics of USS-Graph and the effectiveness of our proposed optimization measures for further knowledge integration and information complementation.

Graph topology enhancement for text classification

Text classification optimization algorithm based on graph neural network

Topological enhanced graph neural networks for semi-supervised node classification

Graph Fusion Network for Text Classification

Enhancing the Resilience of Graph Neural Networks to Topological Perturbations in Sparse Graphs

Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks?

Text Level Graph Neural Network for Text Classification

Contrastive Graph Convolutional Networks with adaptive augmentation for text classification

Tensor Graph Convolutional Networks for Text Classification

Tensor-view Topological Graph Neural Network

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks

Understanding Graph Convolutional Networks for Text Classification

Characterizing the Influence of Topology on Graph Learning Tasks

Graph Neural Networks for Text Classification: A Survey

Hierarchical information matters: Text classification via tree based graph neural network

Deeply Integrating Unsupervised Semantics and Syntax into Heterogeneous Graphs for Inductive Text Classification

An Integrated Graph-of-Words with Tensor Graph Neural Network Learning Paradigm for Text Classification

Topological based classification using graph convolutional networks

Label Incorporated Graph Neural Networks for Text Classification

Heterogeneous Graph Transformer for Meta-structure Learning with Application in Text Classification

Text classification on heterogeneous information network via enhanced GCN and knowledge