Abstract:Text classification is an important and classic application in natural language processing (NLP). Recent studies have shown that graph neural networks (GNNs) are effective in tasks with rich structural relationships and serve as effective transductive learning approaches. Text representation learning methods based on large-scale pretraining can learn implicit but rich semantic information from text. However, few studies have comprehensively utilized the contextual semantic and structural information for Chinese text classification. Moreover, the existing GNN methods for text classification did not consider the applicability of their graph construction methods to long or short texts. In this work, we propose Chinese-BERTology-wwm-GCN, a framework that combines Chinese bidirectional encoder representations from transformers (BERT) series models with whole word masking (Chinese-BERTology-wwm) and the graph convolutional network (GCN) for Chinese text classification. When building text graph, we use documents and words as nodes to construct a heterogeneous graph for the entire corpus. Specifically, we use the term frequency-inverse document frequency (TF-IDF) to construct the word-document edge weights. For long text corpora, we propose an improved pointwise mutual information (PMI*) measure for words according to their word co-occurrence distances to represent the weights of word-word edges. For short text corpora, the co-occurrence information between words is often limited. Therefore, we utilize cosine similarity to represent the word-word edge weights. During the training stage, we effectively combine the cross-entropy and hinge losses and use them to jointly train Chinese-BERTology-wwm and GCN. Experiments show that our proposed framework significantly outperforms the baselines on three Chinese benchmark datasets and achieves good performance even with few labeled training sets.

Extremely Short Chinese Text Classification Method Based on Bidirectional Semantic Extension

Improving short text classification using public search engines

Improving Short Text Classification Through Better Feature Space Selection

A Chinese Short Text Classification Method for Tax Audit Reports based on Word Importance and Syntactic Enhancement BERT

Short Text Classification Based on Strong Feature Thesaurus

Short text classification by detecting information path.

Short text classification based on bidirectional TCN and attention mechanism

Feature-Enhanced Nonequilibrium Bidirectional Long Short-Term Memory Model for Chinese Text Classification

Short Text Classification of Chinese with Label Information Assisting

Key Information Retrieval to Classify the Unstructured Data Content of Preferential Trade Agreements

Combining Lexical and Semantic Features for Short Text Classification.

Feature-enhanced text-inception model for Chinese long text classification

Short Text Classification Improved by Feature Space Extension

Improved Chinese Short Text Classification Method Based on ERNIE_BiGRU Model

Short-Text Classification Detector: A Bert-Based Mental Approach

A multi-semantic passing framework for semi-supervised long text classification

Data Augmentation With Semantic Enrichment for Deep Learning Invoice Text Classification

Chinese text classification by combining Chinese-BERTology-wwm and GCN

Research of Chinese Text Classification Methods Based on Semantic Vector and Semantic Similarity

Chinese text classification method based on sentence information enhancement and feature fusion

An Extended Text Combination Classification Model for Short Video Based on Albert