Abstract:Text classification is an important and classic application in natural language processing (NLP). Recent studies have shown that graph neural networks (GNNs) are effective in tasks with rich structural relationships and serve as effective transductive learning approaches. Text representation learning methods based on large-scale pretraining can learn implicit but rich semantic information from text. However, few studies have comprehensively utilized the contextual semantic and structural information for Chinese text classification. Moreover, the existing GNN methods for text classification did not consider the applicability of their graph construction methods to long or short texts. In this work, we propose Chinese-BERTology-wwm-GCN, a framework that combines Chinese bidirectional encoder representations from transformers (BERT) series models with whole word masking (Chinese-BERTology-wwm) and the graph convolutional network (GCN) for Chinese text classification. When building text graph, we use documents and words as nodes to construct a heterogeneous graph for the entire corpus. Specifically, we use the term frequency-inverse document frequency (TF-IDF) to construct the word-document edge weights. For long text corpora, we propose an improved pointwise mutual information (PMI*) measure for words according to their word co-occurrence distances to represent the weights of word-word edges. For short text corpora, the co-occurrence information between words is often limited. Therefore, we utilize cosine similarity to represent the word-word edge weights. During the training stage, we effectively combine the cross-entropy and hinge losses and use them to jointly train Chinese-BERTology-wwm and GCN. Experiments show that our proposed framework significantly outperforms the baselines on three Chinese benchmark datasets and achieves good performance even with few labeled training sets.

Chinese Text Classification Without Word Segmentation

Chinese Text Categorization Without Word Segmentation Using String Kernel

New Word Identification in Social Network Text Based on Time Series Information

Compositional Recurrent Neural Networks for Chinese Short Text Classification

Text Classification Based on N-gram Language Model

Empirical Study on Character Level Neural Network Classifier for Chinese Text.

Comparing different text representation and feature selection methods on Chinese text classification using Character n-grams

Feature Selection On Chinese Text Classification Using Character N-Grams

Chinese Word Segmentation Without Using Lexicon and Hand-Crafted Training Data

Is Word Segmentation Necessary for Deep Learning of Chinese Representations?

Word-character attention model for Chinese text classification

Hierarchically Classifying Chinese Web Documents Without Dictionary Support And Segmentation Procedure

Chinese Text Classification Using Key Characters String Kernel

Chinese text classification by combining Chinese-BERTology-wwm and GCN

A Chinese text classification model based on radicals and character distinctions

N-grams based feature selection and text representation for Chinese Text Classification

Chinese Image Text Recognition with BLSTM-CTC: A Segmentation-Free Method.

Joint n-gram Chinese language modeling with an application to Chinese word segmentation

Chinese Short Text Classification Based on Interactive Attention Mechanism

A Chinese Character-Level and Word-Level Complementary Text Classification Method.

Hierarchical classification of Chinese documents based onN-grams