Abstract:Text classification is an important and classic application in natural language processing (NLP). Recent studies have shown that graph neural networks (GNNs) are effective in tasks with rich structural relationships and serve as effective transductive learning approaches. Text representation learning methods based on large-scale pretraining can learn implicit but rich semantic information from text. However, few studies have comprehensively utilized the contextual semantic and structural information for Chinese text classification. Moreover, the existing GNN methods for text classification did not consider the applicability of their graph construction methods to long or short texts. In this work, we propose Chinese-BERTology-wwm-GCN, a framework that combines Chinese bidirectional encoder representations from transformers (BERT) series models with whole word masking (Chinese-BERTology-wwm) and the graph convolutional network (GCN) for Chinese text classification. When building text graph, we use documents and words as nodes to construct a heterogeneous graph for the entire corpus. Specifically, we use the term frequency-inverse document frequency (TF-IDF) to construct the word-document edge weights. For long text corpora, we propose an improved pointwise mutual information (PMI*) measure for words according to their word co-occurrence distances to represent the weights of word-word edges. For short text corpora, the co-occurrence information between words is often limited. Therefore, we utilize cosine similarity to represent the word-word edge weights. During the training stage, we effectively combine the cross-entropy and hinge losses and use them to jointly train Chinese-BERTology-wwm and GCN. Experiments show that our proposed framework significantly outperforms the baselines on three Chinese benchmark datasets and achieves good performance even with few labeled training sets.

DC-Graph: a chunk optimization model based on document classification and graph learning

Cognitive Graph for Multi-Hop Reading Comprehension at Scale

Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answering

Billion-scale Pre-Trained Knowledge Graph Model for Conversational Chatbot

Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources

Parallel Fusion of Graph and Text with Semantic Enhancement for Commonsense Question Answering

Improving Question Answering over Knowledge Graphs Using Graph Summarization

End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension

Enhanced Story Comprehension for Large Language Models through Dynamic Document-Based Knowledge Graphs

DocGraphLM: Documental Graph Language Model for Information Extraction

DCQA: Document-Level Chart Question Answering towards Complex Reasoning and Common-Sense Understanding

Grounded Graph Decoding Improves Compositional Generalization in Question Answering

A Step Closer to Comprehensive Answers: Constrained Multi-Stage Question Decomposition with Large Language Models

Chinese text classification by combining Chinese-BERTology-wwm and GCN

Knowledge-aware adaptive graph network for commonsense question answering

DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding

G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering

Meta-Chunking: Learning Efficient Text Segmentation via Logical Perception

Self-DC: When to retrieve and When to generate? Self Divide-and-Conquer for Compositional Unknown Questions

Graph Reasoning for Question Answering with Triplet Retrieval

Enhancing Text-based Knowledge Graph Completion with Zero-Shot Large Language Models: A Focus on Semantic Enhancement