Abstract:Neural network-based approaches have become the driven forces for Natural Language Processing (NLP) tasks. Conventionally, there are two mainstream neural architectures for NLP tasks: the recurrent neural network (RNN) and the convolution neural network (ConvNet). RNNs are good at modeling long-term dependencies over input texts, but preclude parallel computation. ConvNets do not have memory capability and it has to model sequential data as un-ordered features. Therefore, ConvNets fail to learn sequential dependencies over the input texts, but it is able to carry out high-efficient parallel computation. As each neural architecture, such as RNN and ConvNets, has its own pro and con, integration of different architectures is assumed to be able to enrich the semantic representation of texts, thus enhance the performance of NLP tasks. However, few investigation explores the reconciliation of these seemingly incompatible architectures. To address this issue, we propose a hybrid architecture based on a novel hierarchical multi-granularity attention mechanism, named Multi-granularity Attention-based Hybrid Neural Network (MahNN). The attention mechanism is to assign different weights to different parts of the input sequence to increase the computation efficiency and performance of neural models. In MahNN, two types of attentions are introduced: the syntactical attention and the semantical attention. The syntactical attention computes the importance of the syntactic elements (such as words or sentence) at the lower symbolic level and the semantical attention is used to compute the importance of the embedded space dimension corresponding to the upper latent semantics. We adopt the text classification as an exemplifying way to illustrate the ability of MahNN to understand texts. The experimental results on a variety of datasets demonstrate that MahNN outperforms most of the state-of-the-arts for text classification.

A Deep Modular Label Attention Network for Multi-label Text Classification

A Label-Specific Attention-Based Network with Regularized Loss for Multi-label Classification

Research of multi-label text classification based on label attention and correlation networks

Addressing challenges in multi-label text classification: A joint attention and shared semantic space approach

A Label Information Aware Model for Multi-label Text Classification

MsCoa: Multi-Step Co-Attention Model for Multi-Label Classification

Multi-Label Text Classification Model Integrating Label Attention and Historical Attention

HAIN: Multi-label Classification with Hierarchical Attention-based Interaction Network for Multi-turn Dialogue Texts

AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification

Label-Guided Cross-Modal Attention Network for Multi-Label Aerial Image Classification

Deep Multimodal Network for Multi-Label Classification.

MLGN:A Multi-Label Guided Network for Improving Text Classification

Hierarchical Multi-Granularity Attention- Based Hybrid Neural Network for Text Classification.

HAXMLNet: Hierarchical Attention Network for Extreme Multi-Label Text Classification

Spatial Context-Aware Object-Attentional Network for Multi-Label Image Classification

Hierarchical Multi-label Text Classification: Self-adaption Semantic Awareness Network Integrating Text Topic and Label Level Information

Double Attention Based on Graph Attention Network for Image Multi-Label Classification

A Smart Contract Classification Method Based on Label Embedding and Collaborative Attention Mechanism

Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification.

Co-attention network with label embedding for text classification

Label-aware Document Representation via Hybrid Attention for Extreme Multi-Label Text Classification