Abstract:Text classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially pretrained language models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this article, we employ self-supervised learning (SSL) in model learning process and design a novel self-supervised relation of relation (R <sup>2</sup> ) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel () for text classification, in which text classification and R <sup>2</sup> classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multiaspect descriptions for label semantic learning and extend to a novel () from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on contrastive learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that can effectively improve the classification performance and can make better use of label information and further improve the performance. As a byproduct, we have released the codes to facilitate other research.

Knowledge Supervised Text Classification with No Labeled Documents

Knowledge-based Document Embedding for Cross-Domain Text Classification

BoKA: Bayesian Optimization Based Knowledge Amalgamation for Multi-unknown-domain Text Classification

Improving semi-supervised text classification by using wikipedia knowledge

Improving short text classification using public search engines

Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Survey

Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labels

CrowdTC: Crowd-powered Learning for Text Classification

Improving Text Classification Using Knowledge in Labels

Weakly-supervised Text Classification Based on Keyword Graph

Deep Short Text Classification with Knowledge Powered Attention

Open-world Multi-label Text Classification with Extremely Weak Supervision

Teaching Text Classification Models Some Common Sense Via Q&A Statistics: A Light and Transplantable Approach

DOC: Deep Open Classification of Text Documents

Deep learning-based text knowledge classification for whole-process engineering consulting standards

Learning Semantic Topics for Domain-Adapted Textual Knowledge Transfer

Supervised Text Classification using Text Search

Description-Enhanced Label Embedding Contrastive Learning for Text Classification

Dataless Text Classification with Descriptive LDA.

Zero-shot Text Classification via Knowledge Graph Embedding for Social Media Data