Abstract:Neural network-based approaches have become the driven forces for Natural Language Processing (NLP) tasks. Conventionally, there are two mainstream neural architectures for NLP tasks: the recurrent neural network (RNN) and the convolution neural network (ConvNet). RNNs are good at modeling long-term dependencies over input texts, but preclude parallel computation. ConvNets do not have memory capability and it has to model sequential data as un-ordered features. Therefore, ConvNets fail to learn sequential dependencies over the input texts, but it is able to carry out high-efficient parallel computation. As each neural architecture, such as RNN and ConvNets, has its own pro and con, integration of different architectures is assumed to be able to enrich the semantic representation of texts, thus enhance the performance of NLP tasks. However, few investigation explores the reconciliation of these seemingly incompatible architectures. To address this issue, we propose a hybrid architecture based on a novel hierarchical multi-granularity attention mechanism, named Multi-granularity Attention-based Hybrid Neural Network (MahNN). The attention mechanism is to assign different weights to different parts of the input sequence to increase the computation efficiency and performance of neural models. In MahNN, two types of attentions are introduced: the syntactical attention and the semantical attention. The syntactical attention computes the importance of the syntactic elements (such as words or sentence) at the lower symbolic level and the semantical attention is used to compute the importance of the embedded space dimension corresponding to the upper latent semantics. We adopt the text classification as an exemplifying way to illustrate the ability of MahNN to understand texts. The experimental results on a variety of datasets demonstrate that MahNN outperforms most of the state-of-the-arts for text classification.

Knowledge Enhanced Hybrid Neural Network for Text Matching

Incorporating Knowledge into Neural Network for Text Representation.

KETM:A Knowledge-Enhanced Text Matching method

Match-Unity: Long-Form Text Matching With Knowledge Complementarity

The Short Text Matching Model Enhanced with Knowledge via Contrastive Learning

Exploiting Explicit Matching Knowledge With Long Short-Term Memory

HyperMatch: long-form text matching via hypergraph convolutional networks

Incorporating external knowledge for text matching model

Deep Hierarchical Attention Networks for Text Matching in Information Retrieval

Neural Graph Matching Networks for Chinese Short Text Matching.

Matching Long Text Documents Via Graph Convolutional Networks.

Hierarchical Multi-Granularity Attention- Based Hybrid Neural Network for Text Classification.

Multiresolution Graph Attention Networks for Relevance Matching

Attention-Based Multi-level Network for Text Matching with Feature Fusion

LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching

Deep and Shallow Features Learning for Short Texts Matching

Deep Semantic Match Model for Entity Linking Using Knowledge Graph and Text.

Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs

Knowledge Adaptive Multi-Way Matching Network for Biomedical Named Entity Recognition Via Machine Reading Comprehension.

Neural Knowledge Acquisition via Mutual Attention Between Knowledge Graph and Text.

OTE: an Optimized Chinese Short Text Matching Algorithm Based on External Knowledge