Abstract:Machine Comprehension (MC) of text is the problem to answer a query based on a given document. Although MC has been very popular recently, it still have some serious weaknesses which rely only on query-to-document interaction or its learning is just heavily dependent on the training data. To take advantage of external knowledge to improve neural networks for MC, we propose a novel knowledge enhanced recurrent neural model, called knowledge-aware LSTM (k-LSTM), an extension to basic LSTM cells, designed to exploit external knowledge bases (KBs) to improve neural networks for MC task. To incorporate KBs with contextual information effectively from the currently text, k-LSTM employs an compositional attention mechanism to adaptively decide whether to attend to KBs and which information from external knowledge is useful. Furthermore, we present our knowledge enhanced neural network, called Knowledge-guided DIM Reader (K-DIM Reader), which is a novel knowledge-aware compositional attention neural network architecture, employing the k-LSTM in our framework. By stringing external background knowledge together and imposing compositional attention interaction that regulate their interaction, K-DIM Reader effectively learns to perform reading comprehension processes that are directly inferred from the data in an end-to-end approach. We show our proposed models strength, robustness and interpretability on the challenging MC datasets, achieving significant improvements on SQuAD dataset [1] and obtaining new state-of-the-art results on both Cloze-style datasets, CBTest [2] and CNN news [3]. In particular, we further extend 6 popular end-to-end neural MC models using k-LSTM incorporating knowledge into models for improving MC, and evaluate their performance on both well-known MC datasets. We demonstrate that neural model with external knowledge improves performance on MC task.

Enhancing Pre-Trained Language Representations with Rich Knowledge for Machine Reading Comprehension

Machine Reading Comprehension with Rich Knowledge

Improving Biomedical Named Entity Recognition with a Unified Multi-Task MRC Framework

Towards Knowledge Enhanced Language Model for Machine Reading Comprehension

Chinese Machine Reading Comprehension Based on Language Model Containing Knowledge.

Analysis and Improvement of External Knowledge Usage in Machine Multi-Choice Reading Comprehension Tasks

Utilizing External Knowledge with Multi-granularity Attention for Review Reading Comprehension.

Knowledge-Aware LSTM for Machine Comprehension

Enhancing Machine Comprehension Using Multi-Knowledge Bases and Offline Answer Span Improving System

CAT-BERT: A Context-Aware Transferable BERT Model for Multi-turn Machine Reading Comprehension.

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources

KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs

Feeding What You Need by Understanding What You Learned

Enhancing Transformer-based language models with commonsense representations for knowledge-driven machine comprehension

Reference Knowledgeable Network for Machine Reading Comprehension

Knowledge Based Machine Reading Comprehension.

K-BERT: Enabling Language Representation with Knowledge Graph

A Sentence Quality Evaluation Framework for Machine Reading Comprehension Incorporating Pre-trained Language Model.

Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks

Improving the robustness of machine reading comprehension model with hierarchical knowledge and auxiliary unanswerability prediction

Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond