Abstract:Neural network models have been widely used in the field of natural language processing (NLP). Recurrent neural networks (RNNs), which have the ability to process sequences of arbitrary length, are common methods for sequence modeling tasks. Long short-term memory (LSTM) is one kind of RNNs and has achieved remarkable performance in text classification. However, due to the high dimensionality and sparsity of text data, and to the complex semantics of the natural language, text classification presents difficult challenges. In order to solve the above problems, a novel and unified architecture which contains a bidirectional LSTM (BiLSTM), attention mechanism and the convolutional layer is proposed in this paper. The proposed architecture is called attention-based bidirectional long short-term memory with convolution layer (AC-BiLSTM). In AC-BiLSTM, the convolutional layer extracts the higher-level phrase representations from the word embedding vectors and BiLSTM is used to access both the preceding and succeeding context representations. Attention mechanism is employed to give different focus to the information outputted from the hidden layers of BiLSTM. Finally, the softmax classifier is used to classify the processed context information. AC-BiLSTM is able to capture both the local feature of phrases as well as global sentence semantics. Experimental verifications are conducted on six sentiment classification datasets and a question classification dataset, including detailed analysis for AC-BiLSTM. The results clearly show that AC-BiLSTM outperforms other state-of-the-art text classification methods in terms of the classification accuracy.

Bidirectional Gated Temporal Convolution with Attention for text classification

Database Systems for Advanced Applications

Short text classification based on bidirectional TCN and attention mechanism

Attention-based BiLSTM fused CNN with gating mechanism model for Chinese long text classification

Densely Connected CNN with Multi-scale Feature Attention for Text Classification

Local Bidirectional Long Short Term Memory for Text Classification

A text classification method based on a convolutional and bidirectional long short-term memory model

A BERT-Based Hybrid Short Text Classification Model Incorporating CNN and Attention-Based BiGRU

Bidirectional LSTM with attention mechanism and convolutional layer for text classification

Tightly-coupled Convolutional Neural Network with Spatial-Temporal Memory for Text Classification.

CRAN: A Hybrid CNN-RNN Attention-Based Model for Text Classification

Densely Connected Bidirectional LSTM with Max-Pooling of CNN Network for Text Classification.

Text Classification with Attention Gated Graph Neural Network

Novel GCN Model Using Dense Connection and Attention Mechanism for Text Classification

Feature-enhanced text-inception model for Chinese long text classification

A Gated Graph Neural Network With Attention for Text Classification Based on Coupled P Systems

Feature-Enhanced Nonequilibrium Bidirectional Long Short-Term Memory Model for Chinese Text Classification

A convolutional attention model for text classification

A Hybrid Deep Learning Model for Text Classification

Chinese text classification based on attention mechanism and feature-enhanced fusion neural network