Abstract:Graph-based neural networks and unsupervised pre-trained models are both cutting-edge text representation methods, given their outstanding ability to capture global information and contextualized information, respectively. However, both representation methods meet obstacles to further performance improvements. On one hand, graph-based neural networks lack knowledge orientation to guide textual interpretation during global information interaction. On the other hand, unsupervised pre-trained models imply rich semantic and syntactic knowledge which lacks sufficient induction and expression. Therefore, how to effectively integrate graph-based global information and unsupervised contextualized semantic and syntactic information to achieve better text representation is an important issue pending for solution. In this paper, we propose a representation method that deeply integrates Unsupervised Semantics and Syntax into heterogeneous Graphs (USS-Graph) for inductive text classification. By constructing a heterogeneous graph whose edges and nodes are totally generated by knowledge from unsupervised pre-trained models, USS-Graph can harmonize the two perspectives of information under a bidirectionally weighted graph structure and thereby realizing the intra-fusion of graph-based global information and unsupervised contextualized semantic and syntactic information. Based on USS-Graph, we also propose a series of optimization measures to further improve the knowledge integration and representation performance. Extensive experiments conducted on benchmark datasets show that USS-Graph consistently achieves state-of-the-art performances on inductive text classification tasks. Additionally, extended experiments are conducted to deeply analyze the characteristics of USS-Graph and the effectiveness of our proposed optimization measures for further knowledge integration and information complementation.

A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification

A Novel Explainable Structure for Text Classification

Large Language Models are Interpretable Learners

Self-Explaining Structures Improve NLP Models

MLGN:A Multi-Label Guided Network for Improving Text Classification

Hierarchical Interpretation of Neural Text Classification

Interpretable Text Classification Using CNN and Max-pooling

Hierarchical Multi-label Text Classification: Self-adaption Semantic Awareness Network Integrating Text Topic and Label Level Information

LiGCN: Label-interpretable Graph Convolutional Networks for Multi-label Text Classification

Multi-task Learning Using a Hybrid Representation for Text Classification.

Neural Symbolic Representation Learning for Image Captioning

SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers

Interpretable Neural Embeddings with Sparse Self-Representation

Interpretable Structured Multi-modal Deep Neural Network

Leveraging Multi-grained Sentiment Lexicon Information for Neural Sequence Models

SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification

Rethinking Self-Attention: Towards Interpretability in Neural Parsing

Text Classification Using Self-Structure Extended Multinomial Naive Bayes

MFLSCI: Multi-granularity fusion and label semantic correlation information for multi-label legal text classification

Deeply Integrating Unsupervised Semantics and Syntax into Heterogeneous Graphs for Inductive Text Classification

Addressing the Polysemy Problem in Language Modeling with Attentional Multi-Sense Embeddings