Abstract:Graph-based neural networks and unsupervised pre-trained models are both cutting-edge text representation methods, given their outstanding ability to capture global information and contextualized information, respectively. However, both representation methods meet obstacles to further performance improvements. On one hand, graph-based neural networks lack knowledge orientation to guide textual interpretation during global information interaction. On the other hand, unsupervised pre-trained models imply rich semantic and syntactic knowledge which lacks sufficient induction and expression. Therefore, how to effectively integrate graph-based global information and unsupervised contextualized semantic and syntactic information to achieve better text representation is an important issue pending for solution. In this paper, we propose a representation method that deeply integrates Unsupervised Semantics and Syntax into heterogeneous Graphs (USS-Graph) for inductive text classification. By constructing a heterogeneous graph whose edges and nodes are totally generated by knowledge from unsupervised pre-trained models, USS-Graph can harmonize the two perspectives of information under a bidirectionally weighted graph structure and thereby realizing the intra-fusion of graph-based global information and unsupervised contextualized semantic and syntactic information. Based on USS-Graph, we also propose a series of optimization measures to further improve the knowledge integration and representation performance. Extensive experiments conducted on benchmark datasets show that USS-Graph consistently achieves state-of-the-art performances on inductive text classification tasks. Additionally, extended experiments are conducted to deeply analyze the characteristics of USS-Graph and the effectiveness of our proposed optimization measures for further knowledge integration and information complementation.

An Integrated Graph-of-Words with Tensor Graph Neural Network Learning Paradigm for Text Classification

Tensor Graph Convolutional Networks for Text Classification

Graph Neural Networks on Discriminative Graphs of Words

Graph topology enhancement for text classification

Understanding Graph Convolutional Networks for Text Classification

Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks?

Text Level Graph Neural Network for Text Classification

Recursive Graphical Neural Networks for Text Classification

Efficient and effective training of language and graph neural network models

Word Order is Considerable: Contextual Position-aware Graph Neural Network for Text Classification

Tensor-view Topological Graph Neural Network

Text classification optimization algorithm based on graph neural network

InducT-GCN: Inductive Graph Convolutional Networks for Text Classification

Label Incorporated Graph Neural Networks for Text Classification

Text classification on heterogeneous information network via enhanced GCN and knowledge

Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning

GTAGCN: Generalized Topology Adaptive Graph Convolutional Networks

Chinese text classification by combining Chinese-BERTology-wwm and GCN

Deeply Integrating Unsupervised Semantics and Syntax into Heterogeneous Graphs for Inductive Text Classification

GazeGCN: Gaze-aware Graph Convolutional Network for Text Classification

Contrastive Graph Convolutional Networks with adaptive augmentation for text classification