Abstract:Text classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially pretrained language models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this article, we employ self-supervised learning (SSL) in model learning process and design a novel self-supervised relation of relation (R <sup>2</sup> ) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel () for text classification, in which text classification and R <sup>2</sup> classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multiaspect descriptions for label semantic learning and extend to a novel () from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on contrastive learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that can effectively improve the classification performance and can make better use of label information and further improve the performance. As a byproduct, we have released the codes to facilitate other research.

Semi-Supervised Text Classification Via Self-Paced Semantic-Level Contrast

Semi-Supervised Relational Contrastive Learning

DCS: Debiased Contrastive Learning with Weak Supervision for Time Series Classification

Class-Aware Contrastive Semi-Supervised Learning

ASCL: Accelerating semi‐supervised learning via contrastive learning

Semi-supervised Contrastive Learning with Similarity Co-calibration.

Semantic-Enhanced Supervised Contrastive Learning.

CSS-LM: A Contrastive Framework for Semi-Supervised Fine-Tuning of Pre-Trained Language Models

Self Adaptive Threshold Pseudo-labeling and Unreliable Sample Contrastive Loss for Semi-supervised Image Classification

Semi-Supervised End-To-End Contrastive Learning For Time Series Classification

Spatial and Semantic Consistency Contrastive Learning for Self-Supervised Semantic Segmentation of Remote Sensing Images

Investigating Contrastive Pair Learning's Frontiers in Supervised, Semisupervised, and Self-Supervised Learning

Interpolation-Based Contrastive Learning for Few-Label Semi-Supervised Learning

Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification

Description-Enhanced Label Embedding Contrastive Learning for Text Classification

Dual Pseudo Supervision for Semi-Supervised Text Classification with a Reliable Teacher

Semi-Supervised Semantic Segmentation Via Dynamic Self-Training and Class-Balanced Curriculum.

Alleviating Over-smoothing for Unsupervised Sentence Representation.

Contrastive classification: A label-independent generalization model for text classification