Abstract:Neural network-based approaches have become the driven forces for Natural Language Processing (NLP) tasks. Conventionally, there are two mainstream neural architectures for NLP tasks: the recurrent neural network (RNN) and the convolution neural network (ConvNet). RNNs are good at modeling long-term dependencies over input texts, but preclude parallel computation. ConvNets do not have memory capability and it has to model sequential data as un-ordered features. Therefore, ConvNets fail to learn sequential dependencies over the input texts, but it is able to carry out high-efficient parallel computation. As each neural architecture, such as RNN and ConvNets, has its own pro and con, integration of different architectures is assumed to be able to enrich the semantic representation of texts, thus enhance the performance of NLP tasks. However, few investigation explores the reconciliation of these seemingly incompatible architectures. To address this issue, we propose a hybrid architecture based on a novel hierarchical multi-granularity attention mechanism, named Multi-granularity Attention-based Hybrid Neural Network (MahNN). The attention mechanism is to assign different weights to different parts of the input sequence to increase the computation efficiency and performance of neural models. In MahNN, two types of attentions are introduced: the syntactical attention and the semantical attention. The syntactical attention computes the importance of the syntactic elements (such as words or sentence) at the lower symbolic level and the semantical attention is used to compute the importance of the embedded space dimension corresponding to the upper latent semantics. We adopt the text classification as an exemplifying way to illustrate the ability of MahNN to understand texts. The experimental results on a variety of datasets demonstrate that MahNN outperforms most of the state-of-the-arts for text classification.

Hierarchical Attention Prototypical Networks for Few-Shot Text Classification.

Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification.

Cross Attention Network for Few-shot Classification

Enhance Prototypical Network with Hybrid Loss for Few-Shot Text Classification

Hierarchical Multi-Granularity Attention- Based Hybrid Neural Network for Text Classification.

Modified Prototypical Networks for Few-Shot Text Classification Based on Class-Covariance Metric and Attention

Hybrid Contrastive Prototypical Network for Few-Shot Scene Classification

Self-similarity Feature Based Few-Shot Learning Via Hierarchical Relation Network.

Dynamic Prototype Selection By Fusing Attention Mechanism For Few-Shot Relation Classification

Self-Adaptive Embedding for Few-Shot Classification by Hierarchical Attention.

Attribute- and attention-guided few-shot classification

Hierarchical Representation based Query-Specific Prototypical Network for Few-Shot Image Classification

Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network Approach

Few-Shot Text Classification with Edge-Labeling Graph Neural Network-Based Prototypical Network.

Selectively Augmented Attention Network for Few-Shot Image Classification

Multi-instance attention network for few-shot learning

Few-Shot Image Classification Based on Asymmetric Convolution and Attention Mechanism

Few-shot learning based on hierarchical feature fusion via relation networks

Feature Interaction Prototypical Network for Few-Shot Fine-Grained Image Classification

Hierarchical Attention Networks for Hyperspectral Image Classification

Few-shot Text Classification by Leveraging Bi-Directional Attention and Cross-Class Knowledge