Abstract:Text classification is an important and classic application in natural language processing (NLP). Recent studies have shown that graph neural networks (GNNs) are effective in tasks with rich structural relationships and serve as effective transductive learning approaches. Text representation learning methods based on large-scale pretraining can learn implicit but rich semantic information from text. However, few studies have comprehensively utilized the contextual semantic and structural information for Chinese text classification. Moreover, the existing GNN methods for text classification did not consider the applicability of their graph construction methods to long or short texts. In this work, we propose Chinese-BERTology-wwm-GCN, a framework that combines Chinese bidirectional encoder representations from transformers (BERT) series models with whole word masking (Chinese-BERTology-wwm) and the graph convolutional network (GCN) for Chinese text classification. When building text graph, we use documents and words as nodes to construct a heterogeneous graph for the entire corpus. Specifically, we use the term frequency-inverse document frequency (TF-IDF) to construct the word-document edge weights. For long text corpora, we propose an improved pointwise mutual information (PMI*) measure for words according to their word co-occurrence distances to represent the weights of word-word edges. For short text corpora, the co-occurrence information between words is often limited. Therefore, we utilize cosine similarity to represent the word-word edge weights. During the training stage, we effectively combine the cross-entropy and hinge losses and use them to jointly train Chinese-BERTology-wwm and GCN. Experiments show that our proposed framework significantly outperforms the baselines on three Chinese benchmark datasets and achieves good performance even with few labeled training sets.

Text Classification Method Based on Bidirectional Attention and Gated Graph Convolutional Networks

Text Classification Based on Attention Gated Graph Neural Network

Text Classification with Attention Gated Graph Neural Network

3D Point Cloud Classification Method Based on Multiple Attention Mechanism and Dynamic Graph Convolution

Graph Convolution Word Embedding and Attention for Text Classification∗

The Study on the Text Classification Based on Graph Convolutional Network and BiLSTM

Bidirectional Gated Temporal Convolution with Attention for text classification

A Gating Context-Aware Text Classification Model with BERT and Graph Convolutional Networks

Text Classification Based on Knowledge Graphs and Improved Attention Mechanism

A Heterogeneous Directed Graph Attention Network for Inductive Text Classification Using Multilevel Semantic Embeddings

A Graph Neural Network Fused with Multi-Head Attention for Text Classification

A text classification method based on a convolutional and bidirectional long short-term memory model

KGAT: An Enhanced Graph-Based Model for Text Classification

A Gated Graph Neural Network With Attention for Text Classification Based on Coupled P Systems

Text Classification Method Based on BiGRU-Attention and CNN Hybrid Model

Attention-based BiLSTM fused CNN with gating mechanism model for Chinese long text classification

Deep Attention Diffusion Graph Neural Networks for Text Classification.

Text Classification Method Based on Information Fusion of Dual-graph Neural Network

A gated graph attention network based on dual graph convolution for node embedding

An Integration Model for Text Classification Using Graph Convolutional Network and BERT

Chinese text classification by combining Chinese-BERTology-wwm and GCN