Abstract:The goal of text classification is to identify the category to which the text belongs. Text categorization is widely used in email detection, sentiment analysis, topic marking and other fields. However, good text representation is the point to improve the capability of NLP tasks. Traditional text representation adopts bag-of-words model or vector space model, which loses the context information of the text and faces the problems of high latitude and high sparsity,. In recent years, with the increase of data and the improvement of computing performance, the use of deep learning technology to represent and classify texts has attracted great attention. Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and RNN with attention mechanism are used to represent the text, and then to classify the text and other NLP tasks, all of which have better performance than the traditional methods. In this paper, we design two sentence-level models based on the deep network and the details are as follows: (1) Text representation and classification model based on bidirectional RNN and CNN (BRCNN). BRCNN’s input is the word vector corresponding to each word in the sentence; after using RNN to extract word order information in sentences, CNN is used to extract higher-level features of sentences. After convolution, the maximum pool operation is used to obtain sentence vectors. At last, softmax classifier is used for classification. RNN can capture the word order information in sentences, while CNN can extract useful features. Experiments on eight text classification tasks show that BRCNN model can get better text feature representation, and the classification accuracy rate is equal to or higher than that of the prior art. (2) Attention mechanism and CNN (ACNN) model uses the RNN with attention mechanism to obtain the context vector; Then CNN is used to extract more advanced feature information. The maximum pool operation is adopted to obtain a sentence vector; At last, the softmax classifier is used to classify the text. Experiments on eight text classification benchmark data sets show that ACNN improves the stability of model convergence, and can converge to an optimal or local optimal solution better than BRCNN.

Self-Attention-Based Convolutional Neural Networks for Sentence Classification

An Attention-Gated Convolutional Neural Network for Sentence Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

SAC: Accelerating and Structuring Self-Attention Via Sparse Adaptive Connection.

SECNN: Squeeze-and-Excitation Convolutional Neural Network for Sentence Classification

Self-Attention Capsule Networks for Object Classification

Convolutional Neural Networks for Sentence Classification

Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring.

Multichannel CNN with Attention for Text Classification

TextConvoNet: a convolutional neural network based architecture for text classification

Multi-channel Convolutional Neural Network with Sentiment Information for Sentiment Classification

Three-way enhanced convolutional neural networks for sentence-level sentiment classification

CRAN: A Hybrid CNN-RNN Attention-Based Model for Text Classification

Self-Convolution for Automatic Speech Recognition

An Efficient Cross-lingual Model for Sentence Classification Using Convolutional Neural Network.

A Deep Neural Network for Audio Classification with a Classifier Attention Mechanism

Classify Sentence From Multiple Perspectives With Category Expert Attention Network

Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms

HAM: Hybrid Attention Module in Deep Convolutional Neural Networks for Image Classification

Text Classification Based on Neural Network Fusion

Self-Attention Causal Dilated Convolutional Neural Network for Multivariate Time Series Classification and Its Application