Abstract:Text classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially pretrained language models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this article, we employ self-supervised learning (SSL) in model learning process and design a novel self-supervised relation of relation (R <sup>2</sup> ) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel () for text classification, in which text classification and R <sup>2</sup> classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multiaspect descriptions for label semantic learning and extend to a novel () from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on contrastive learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that can effectively improve the classification performance and can make better use of label information and further improve the performance. As a byproduct, we have released the codes to facilitate other research.

A Smart Contract Classification Method Based on Label Embedding and Collaborative Attention Mechanism

Towards Automatic Smart-contract Codes Classification by Means of Word Embedding Model and Transaction Information

CVAE-Attention: CVAE based Semi-Supervised Sentiment Classification using Attention

A Multi-feature Fusion Method for Smart Contract Classification

Towards Automated Reentrancy Detection for Smart Contracts Based on Sequential Models

Co-attention network with label embedding for text classification

Cross-Modality Mutual Learning for Enhancing Smart Contract Vulnerability Detection on Bytecode

A Bytecode-based Approach for Smart Contract Classification

MFF-SC: A multi-feature fusion method for smart contract classification

Multi-label classification of legal text based on label embedding and capsule network

Addressing challenges in multi-label text classification: A joint attention and shared semantic space approach

A Label-Specific Attention-Based Network with Regularized Loss for Multi-label Classification

An ABI-based Classification Approach for Ethereum Smart Contracts

Label-Attentive Hierarchical Attention Network for Text Classification

Label-aware Document Representation via Hybrid Attention for Extreme Multi-Label Text Classification

K-Nearest neighbor smart contract classification with semantic feature enhancement

Transaction-based classification and detection approach for Ethereum smart contract

Smart Contract Vulnerability Detection Based on Multimodal Feature Fusion

Attention Mechanisms in Clinical Text Classification: A Comparative Evaluation

Exploiting Smart Contract Bytecode for Classification on Ethereum

Description-Enhanced Label Embedding Contrastive Learning for Text Classification