Abstract:Text classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially pretrained language models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this article, we employ self-supervised learning (SSL) in model learning process and design a novel self-supervised relation of relation (R <sup>2</sup> ) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel () for text classification, in which text classification and R <sup>2</sup> classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multiaspect descriptions for label semantic learning and extend to a novel () from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on contrastive learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that can effectively improve the classification performance and can make better use of label information and further improve the performance. As a byproduct, we have released the codes to facilitate other research.

Bag-of-Embeddings for Text Classification.

Knowledge-based Document Embedding for Cross-Domain Text Classification

Text Classification With Document Embeddings

Word Embedding Composition for Data Imbalances in Sentiment and Emotion Classification

Continuous-bag-of-words and Skip-gram for word vector training and text classification

Semi-Supervised Multinomial Naive Bayes for Text Classification by Leveraging Word-Level Statistical Constraint

Chinese Short Text Multi-Classification Based on Word and Part-of-Speech Tagging Embedding

Chinese text classification by combining Chinese-BERTology-wwm and GCN

Description-Enhanced Label Embedding Contrastive Learning for Text Classification

A Deep Learning Short Text Classification Model Integrating Part of Speech Features

Chinese Text Classification Based on the BVB Model

Adaptive Region Embedding for Text Classification.

A text classification method based on a convolutional and bidirectional long short-term memory model

Feature-Enhanced Nonequilibrium Bidirectional Long Short-Term Memory Model for Chinese Text Classification

Word-class embeddings for multiclass text classification

Word Vector Enrichment of Low Frequency Words in the Bag-of-Words Model for Short Text Multi-class Classification Problems

Chinese text classification method based on sentence information enhancement and feature fusion

Few-Shot Transfer Learning for Text Classification With Lightweight Word Embedding Based Models

Feature-enhanced text-inception model for Chinese long text classification

DCNN-BiGRU Text Classification Model Based on BERT Embedding

Bagging Text Classification Algorithm Based on Classifier Performance Evaluation