Abstract:Graph-based neural networks and unsupervised pre-trained models are both cutting-edge text representation methods, given their outstanding ability to capture global information and contextualized information, respectively. However, both representation methods meet obstacles to further performance improvements. On one hand, graph-based neural networks lack knowledge orientation to guide textual interpretation during global information interaction. On the other hand, unsupervised pre-trained models imply rich semantic and syntactic knowledge which lacks sufficient induction and expression. Therefore, how to effectively integrate graph-based global information and unsupervised contextualized semantic and syntactic information to achieve better text representation is an important issue pending for solution. In this paper, we propose a representation method that deeply integrates Unsupervised Semantics and Syntax into heterogeneous Graphs (USS-Graph) for inductive text classification. By constructing a heterogeneous graph whose edges and nodes are totally generated by knowledge from unsupervised pre-trained models, USS-Graph can harmonize the two perspectives of information under a bidirectionally weighted graph structure and thereby realizing the intra-fusion of graph-based global information and unsupervised contextualized semantic and syntactic information. Based on USS-Graph, we also propose a series of optimization measures to further improve the knowledge integration and representation performance. Extensive experiments conducted on benchmark datasets show that USS-Graph consistently achieves state-of-the-art performances on inductive text classification tasks. Additionally, extended experiments are conducted to deeply analyze the characteristics of USS-Graph and the effectiveness of our proposed optimization measures for further knowledge integration and information complementation.

Self-supervised Short Text Classification with Heterogeneous Graph Neural Networks

HGAT: Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

Contrastive Learning with Heterogeneous Graph Attention Networks on Short Text Classification

Heterogeneous graph contrastive learning with adaptive data augmentation for semi‐supervised short text classification

Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification.

Heterogeneous Graph Representation for Text Mining

A Word-Concept Heterogeneous Graph Convolutional Network for Short Text Classification

Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning

A Graph Neural Network with Type-Feature Attention for Node Classification on Heterogeneous Graphs

Cross-lingual Text Classification with Heterogeneous Graph Neural Network

Heterogeneous Graph Transformer for Meta-structure Learning with Application in Text Classification

HetGNN-SF: Self-supervised learning on heterogeneous graph neural network via semantic strength and feature similarity

Text classification on heterogeneous information network via enhanced GCN and knowledge

Self-attention Dual Embedding for Graphs with Heterophily

Deeply Integrating Unsupervised Semantics and Syntax into Heterogeneous Graphs for Inductive Text Classification

Towards Self-supervised Learning on Graphs with Heterophily

Self-supervised Nodes-Hyperedges Embedding for Heterogeneous Information Network Learning

Heterogeneous Hypergraph Embedding for Graph Classification

Contrastive Multi-graph Learning with Neighbor Hierarchical Sifting for Semi-supervised Text Classification

Edge-enhanced minimum-margin graph attention network for short text classification