Abstract:Neural network-based approaches have become the driven forces for Natural Language Processing (NLP) tasks. Conventionally, there are two mainstream neural architectures for NLP tasks: the recurrent neural network (RNN) and the convolution neural network (ConvNet). RNNs are good at modeling long-term dependencies over input texts, but preclude parallel computation. ConvNets do not have memory capability and it has to model sequential data as un-ordered features. Therefore, ConvNets fail to learn sequential dependencies over the input texts, but it is able to carry out high-efficient parallel computation. As each neural architecture, such as RNN and ConvNets, has its own pro and con, integration of different architectures is assumed to be able to enrich the semantic representation of texts, thus enhance the performance of NLP tasks. However, few investigation explores the reconciliation of these seemingly incompatible architectures. To address this issue, we propose a hybrid architecture based on a novel hierarchical multi-granularity attention mechanism, named Multi-granularity Attention-based Hybrid Neural Network (MahNN). The attention mechanism is to assign different weights to different parts of the input sequence to increase the computation efficiency and performance of neural models. In MahNN, two types of attentions are introduced: the syntactical attention and the semantical attention. The syntactical attention computes the importance of the syntactic elements (such as words or sentence) at the lower symbolic level and the semantical attention is used to compute the importance of the embedded space dimension corresponding to the upper latent semantics. We adopt the text classification as an exemplifying way to illustrate the ability of MahNN to understand texts. The experimental results on a variety of datasets demonstrate that MahNN outperforms most of the state-of-the-arts for text classification.

Question-Answering Aspect Classification with Hierarchical Attention Network.

Question-Answering Aspect Classification With Multi-Attention Representation

Multi-Label Aspect Classification On Question-Answering Text With Contextualized Attention-Based Neural Network

Sentiment Classification Towards Question-Answering with Hierarchical Matching Network

Aspect Sentiment Classification Towards Question-Answering with Reinforced Bidirectional Attention Network.

A Relative Position Attention Network for Aspect-Based Sentiment Analysis

Aspect Sentiment Classification with Both Word-level and Clause-level Attention Networks.

Aspect Category Detection Based on Attention Mechanism and Bi-Directional LSTM

Hierarchical Multi-Granularity Attention- Based Hybrid Neural Network for Text Classification.

Aspect-Based Sentiment Analysis on Convolution Neural Network and Multi-Hierarchical Attention.

Video Question Answering Via Hierarchical Dual-Level Attention Network Learning.

Module s 1 , q k s 2 , q k s T , q k Hierarchical

Type-Aware Question Answering over Knowledge Base with Attention-Based Tree-Structured Neural Networks

Aspect Level Sentiment Classification with Attention-over-Attention Neural Networks

Video Question Answering Via Grounded Cross-Attention Network Learning.

Hierarchical Attention Networks for Multimodal Machine Learning

Hierarchical Attention Networks for Image Classification of Remote Sensing Images Based on Visual Q&A Methods

Hierarchical Question-Image Co-Attention for Visual Question Answering

A Question-Focused Multi-Factor Attention Network for Question Answering

Semi-interactive Attention Network for Answer Understanding in Reverse-QA

Hierarchical Temporal Fusion of Multi-grained Attention Features for Video Question Answering