Abstract:Recurrent Neural Network (RNN) is one of the most popular architectures used in Natural Language Processsing (NLP) tasks because its recurrent structure is very suitable to process variable-length text. RNN can utilize distributed representations of words by first converting the tokens comprising each text into vectors, which form a matrix. And this matrix includes two dimensions: the time-step dimension and the feature vector dimension. Then most existing models usually utilize one-dimensional (1D) max pooling operation or attention-based operation only on the time-step dimension to obtain a fixed-length vector. However, the features on the feature vector dimension are not mutually independent, and simply applying 1D pooling operation over the time-step dimension independently may destroy the structure of the feature representation. On the other hand, applying two-dimensional (2D) pooling operation over the two dimensions may sample more meaningful features for sequence modeling tasks. To integrate the features on both dimensions of the matrix, this paper explores applying 2D max pooling operation to obtain a fixed-length representation of the text. This paper also utilizes 2D convolution to sample more meaningful information of the matrix. Experiments are conducted on six text classification tasks, including sentiment analysis, question classification, subjectivity classification and newsgroup classification. Compared with the state-of-the-art models, the proposed models achieve excellent performance on 4 out of 6 tasks. Specifically, one of the proposed models achieves highest accuracy on Stanford Sentiment Treebank binary classification and fine-grained classification tasks.

Combining Attention-Based Bidirectional Gated Recurrent Neural Network and Two-Dimensional Convolutional Neural Network for Document-Level Sentiment Classification

Document-Level Sentiment Analysis Using Attention-Based Bi-Directional Long Short-Term Memory Network and Two-Dimensional Convolutional Neural Network

Document Modeling with Gated Recurrent Neural Network for Sentiment Classification.

Combining Gated Recurrent Unit and Attention Pooling for Sentimental Classification.

Bidirectional convolutional recurrent neural network architecture with group-wise enhancement mechanism for text sentiment classification

HieNN-DWE: A Hierarchical Neural Network with Dynamic Word Embeddings for Document Level Sentiment Classification

Attention Based BiGRU-2DCNN with Hunger Game Search Technique for Low-Resource Document-Level Sentiment Classification

Sequential Multi-Kernel Convolutional Recurrent Network for Sentiment Classification

Attention-based ResNet for Chinese Text Sentiment Classification

Text Sentiment Analysis Based on Convolutional Neural Network and Bidirectional LSTM Model.

Sentiment Classification Using Comprehensive Attention Recurrent Models

LSTM with sentence representations for document-level sentiment classification.

Attention-based Hierarchical LSTM Model for Document Sentiment Classification

Convolution-Based Neural Attention with Applications to Sentiment Classification

A Transfer Learning Based Hierarchical Attention Neural Network for Sentiment Classification.

Feature Fusion Text Classification Model Combining CNN and BiGRU with Multi-Attention Mechanism

Combining Vector Space Features and Convolution Neural Network for Text Sentiment Analysis.

Hierarchical Hybrid Neural Networks with Multi-Head Attention for Document Classification

Hierarchical Recurrent and Convolutional Neural Network Based on Attention for Chinese Document Classification

A Hierarchical Neural-Network-Based Document Representation Approach for Text Classification

Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling