Abstract:Convolutional neural network has been proven to be a powerful semantic composition model for modelling sentences. A standard convolutional neural network usually consists of several convolutional and pooling layers at the bottom of a linear or non-linear classifier. In this paper, a new pooling scheme termed Attention Pooling is proposed to retain the most significant information at the pooling stage. An intermediate sentence representation generated by the bidirectional long short-term memory is used as a reference for local representations produced by the convolutional layer to obtain attention weights. The sentence representation is formed by combining local representations using obtained attention weights. The intermediate sentence representation is used as an input to the top classifier as well in the testing phase. The salient features of the proposed attention pooling-based convolutional neural network are: (1) The model can be trained end-to-end with limited hyper-parameters; (2) Comprehensive information is extracted by the new pooling scheme and the combination of the convolutional layer and the bidirectional long-short term memory; (3) The model can implicitly separate the sentences from different classes. Experimental results demonstrate that the new model outperforms the state-of-the-art approaches on seven benchmark datasets for text classification. The learning capability of the proposed method is greatly improved and the classification accuracy is even enhanced significantly by over 2% on some datasets. The robustness of the proposed model is evidenced by some statistical tests.

Modularized Syntactic Neural Networks for Sentence Classification.

CMNN: Coupled Modular Neural Network.

Modular Tree Network for Source Code Representation Learning

Discriminative Neural Sentence Modeling by Tree-Based Convolution

On Tree-Based Neural Sentence Modeling

Tree-based Convolution for Sentence Modeling.

Tree-based convolution: A new architecture for sentence modeling

Modularity Facilitates Classification Performance of Spiking Neural Networks for Decoding Cortical Spike Trains

Encoding Syntactic Knowledge in Neural Networks for Sentiment Classification.

Modular neural network via exploring category hierarchy

Syntax-Based Context Representation for Statistical Machine Translation.

Syntax-aware Neural Semantic Role Labeling

Tree-LSTM Guided Attention Pooling of DCNN for Semantic Sentence Modeling.

AHNN: an Attention-Based Hybrid Neural Network for Sentence Modeling

Shortcut Enhanced Syntactic and Semantic Dual-channel Network for Aspect-based Sentiment Analysis

Efficient Part-of-Speech Tagging with a Min-Max Modular Neural-Network Model

Sentiment Classification Based on Syntax Tree Pruning and Tree Kernel

Text-dependent speaker identification based on the modular tree

Attention Pooling-Based Convolutional Neural Network for Sentence Modelling

A Convolutional Neural Network for Modelling Sentences