Abstract:Text classification is an important and classic application in natural language processing (NLP). Recent studies have shown that graph neural networks (GNNs) are effective in tasks with rich structural relationships and serve as effective transductive learning approaches. Text representation learning methods based on large-scale pretraining can learn implicit but rich semantic information from text. However, few studies have comprehensively utilized the contextual semantic and structural information for Chinese text classification. Moreover, the existing GNN methods for text classification did not consider the applicability of their graph construction methods to long or short texts. In this work, we propose Chinese-BERTology-wwm-GCN, a framework that combines Chinese bidirectional encoder representations from transformers (BERT) series models with whole word masking (Chinese-BERTology-wwm) and the graph convolutional network (GCN) for Chinese text classification. When building text graph, we use documents and words as nodes to construct a heterogeneous graph for the entire corpus. Specifically, we use the term frequency-inverse document frequency (TF-IDF) to construct the word-document edge weights. For long text corpora, we propose an improved pointwise mutual information (PMI*) measure for words according to their word co-occurrence distances to represent the weights of word-word edges. For short text corpora, the co-occurrence information between words is often limited. Therefore, we utilize cosine similarity to represent the word-word edge weights. During the training stage, we effectively combine the cross-entropy and hinge losses and use them to jointly train Chinese-BERTology-wwm and GCN. Experiments show that our proposed framework significantly outperforms the baselines on three Chinese benchmark datasets and achieves good performance even with few labeled training sets.

Two-stage three-way enhanced technique for ensemble learning in inclusive policy text classification

Exploring Ensemble Oversampling Method for Imbalanced Keyword Extraction Learning in Policy Text Based on Three-Way Decisions and SMOTE

Weak-PMLC: A large-scale framework for multi-label policy classification based on extremely weak supervision

Social-Correlation Based Mutual Reinforcement for Short Text Classification and User Interest Tagging

Adaptive Bi-Encoder Model Selection and Ensemble for Text Classification

Three-way enhanced convolutional neural networks for sentence-level sentiment classification

Machine learning based software effort estimation using development-centric features for crowdsourcing platform

Assessing the Influence Level of Food Safety Public Opinion with Unbalanced Samples Using Ensemble Machine Learning

Hybrid Decision Based Chinese News Headline Classification

Two-Phase Classification Based On Three-Way Decisions

A dual-ways feature fusion mechanism enhancing active learning based on TextCNN

MuTCELM: An optimal multi-TextCNN-based ensemble learning for text classification

A Machine Learning Based Ensemble Method for Automatic Multiclass Classification of Decisions

Chinese text classification by combining Chinese-BERTology-wwm and GCN

Improving imbalance classification via ensemble learning based on two-stage learning

Improving Generalization Ability of Instance Transfer-Based Imbalanced Sentiment Classification of Turn-Level Interactive Chinese Texts

E3WD: A Three-Way Decision Model Based on Ensemble Learning

Text classification based on ensemble extreme learning machine

An Ensemble Learning Approach for Urban Land Use Mapping Based on Remote Sensing Imagery and Social Sensing Data

A multi-level classification based ensemble and feature extractor for credit risk assessment

Comprehensive Implementation of TextCNN for Enhanced Collaboration between Natural Language Processing and System Recommendation