Abstract:Graph-based neural networks and unsupervised pre-trained models are both cutting-edge text representation methods, given their outstanding ability to capture global information and contextualized information, respectively. However, both representation methods meet obstacles to further performance improvements. On one hand, graph-based neural networks lack knowledge orientation to guide textual interpretation during global information interaction. On the other hand, unsupervised pre-trained models imply rich semantic and syntactic knowledge which lacks sufficient induction and expression. Therefore, how to effectively integrate graph-based global information and unsupervised contextualized semantic and syntactic information to achieve better text representation is an important issue pending for solution. In this paper, we propose a representation method that deeply integrates Unsupervised Semantics and Syntax into heterogeneous Graphs (USS-Graph) for inductive text classification. By constructing a heterogeneous graph whose edges and nodes are totally generated by knowledge from unsupervised pre-trained models, USS-Graph can harmonize the two perspectives of information under a bidirectionally weighted graph structure and thereby realizing the intra-fusion of graph-based global information and unsupervised contextualized semantic and syntactic information. Based on USS-Graph, we also propose a series of optimization measures to further improve the knowledge integration and representation performance. Extensive experiments conducted on benchmark datasets show that USS-Graph consistently achieves state-of-the-art performances on inductive text classification tasks. Additionally, extended experiments are conducted to deeply analyze the characteristics of USS-Graph and the effectiveness of our proposed optimization measures for further knowledge integration and information complementation.

TextGTL: Graph-based Transductive Learning for Semi-supervised Text Classification Via Structure-Sensitive Interpolation

Heterogeneous Graph Transformer for Meta-structure Learning with Application in Text Classification

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Tensor Graph Convolutional Networks for Text Classification

Deeply Integrating Unsupervised Semantics and Syntax into Heterogeneous Graphs for Inductive Text Classification

TGNN: A Joint Semi-supervised Framework for Graph-level Classification

Graph topology enhancement for text classification

SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning

Contrastive Graph Convolutional Networks with adaptive augmentation for text classification

TTG-Text: A Graph-Based Text Representation Framework Enhanced by Typical Testors for Improved Classification

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks

Time-aware Graph Structure Learning Via Sequence Prediction on Temporal Graphs.

Contrastive Multi-graph Learning with Neighbor Hierarchical Sifting for Semi-supervised Text Classification

HGAT: Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

Heterogeneous graph contrastive learning with adaptive data augmentation for semi‐supervised short text classification

Text classification on heterogeneous information network via enhanced GCN and knowledge

A Fully Test-Time Training Framework for Semi-Supervised Node Classification on Out-of-Distribution Graphs

Graph Contrastive Learning via Cluster-refined Negative Sampling for Semi-supervised Text Classification

Rethinking the Promotion Brought by Contrastive Learning to Semi-Supervised Node Classification

Contrastive Learning with Heterogeneous Graph Attention Networks on Short Text Classification

Exploring Structure-Adaptive Graph Learning for Robust Semi-Supervised Classification