Abstract:Neural network-based approaches have become the driven forces for Natural Language Processing (NLP) tasks. Conventionally, there are two mainstream neural architectures for NLP tasks: the recurrent neural network (RNN) and the convolution neural network (ConvNet). RNNs are good at modeling long-term dependencies over input texts, but preclude parallel computation. ConvNets do not have memory capability and it has to model sequential data as un-ordered features. Therefore, ConvNets fail to learn sequential dependencies over the input texts, but it is able to carry out high-efficient parallel computation. As each neural architecture, such as RNN and ConvNets, has its own pro and con, integration of different architectures is assumed to be able to enrich the semantic representation of texts, thus enhance the performance of NLP tasks. However, few investigation explores the reconciliation of these seemingly incompatible architectures. To address this issue, we propose a hybrid architecture based on a novel hierarchical multi-granularity attention mechanism, named Multi-granularity Attention-based Hybrid Neural Network (MahNN). The attention mechanism is to assign different weights to different parts of the input sequence to increase the computation efficiency and performance of neural models. In MahNN, two types of attentions are introduced: the syntactical attention and the semantical attention. The syntactical attention computes the importance of the syntactic elements (such as words or sentence) at the lower symbolic level and the semantical attention is used to compute the importance of the embedded space dimension corresponding to the upper latent semantics. We adopt the text classification as an exemplifying way to illustrate the ability of MahNN to understand texts. The experimental results on a variety of datasets demonstrate that MahNN outperforms most of the state-of-the-arts for text classification.

Ontology construction and mapping of multi-source heterogeneous data based on hybrid neural network and autoencoder

Incorporating Knowledge into Neural Network for Text Representation.

An Ontology Mapping Architecture To Facilitate Semantic Integration

Hierarchical Multi-Granularity Attention- Based Hybrid Neural Network for Text Classification.

Geographic categories mapping based on ontology attribute characteristics learning

A Novel Automatic Ontology Construction Method Based on Web Data

Multi-source and Heterogeneous Data Aggregation Method for Power Transmission and Transformation Equipment Panoramic Information

Hybrid Deep Neural Network-Based Cross-Modal Image and Text Retrieval Method for Large-Scale Data.

Matching Biomedical Ontologies via a Hybrid Graph Attention Network

Text Classification Based On A Combination Of Ontology With Statistical Method

Neural Architecture Search Via Multi-Hashing Embedding and Graph Tensor Networks for Multilingual Text Classification

Cross-modal Semantic Autoencoder with Embedding Consensus

Deeply Integrating Unsupervised Semantics and Syntax into Heterogeneous Graphs for Inductive Text Classification

Pre-Trained Multi-View Word Embedding Using Two-Side Neural Network

MEDTO: Medical Data to Ontology Matching Using Hybrid Graph Neural Networks

Towards a Semi-Automatic Ontology Mapping - An Approach Using Instance Based Learning and Logic Relation Mining

A semi-supervised approach for the integration of multi-omics data based on transformer multi-head self-attention mechanism and graph convolutional networks

Research on Ontology-Based Data Fusion.

A Multi-Source Heterogeneous Data Fusion Method Based on Deep Learning

A Multi-Stage Strategy for Ontology Mapping Resolution

Ontology-Based Automatic Mapping Technology for Heterogeneous Common Information Model