Abstract:Knowledge-based question answering (KBQA) is a key task in NLP research, and also an approach to access the web data and knowledge, which requires exploiting knowledge graphs (KGs) for reasoning. In the literature, one promising solution for KBQA is to incorporate the pretrained language model (LM) with KGs by generating KG-centered pretraining corpus, which has shown its superiority. However, these methods often depend on specific techniques and resources to work, which may not always be available and restrict its application. Moreover, existing methods focus more on improving language understanding with KGs, while neglect the more important human-like complex reasoning. To this end, in this paper, we propose a general Knowledge-Injected Curriculum Pretraining framework (KICP) to achieve comprehensive KG learning and exploitation for KBQA tasks, which is composed of knowledge injection (KI), knowledge adaptation (KA) and curriculum reasoning (CR). Specifically, the KI module first injects knowledge into the LM by generating KG-centered pretraining corpus, and generalizes the process into three key steps that could work with different implementations for flexible application. Next, the KA module learns knowledge from the generated corpus with LM equipped with an adapter as well as keeps its original natural language understanding ability to reduce the negative impacts of the difference between the generated and natural corpus. Last, to enable the LM with complex reasoning, the CR module follows human reasoning patterns to construct three corpora with increasing difficulties of reasoning, and further trains the LM from easy to hard in a curriculum manner. We provide an implementation of the general framework, and evaluate the proposed KICP on four real-word datasets. The results demonstrate that our framework can achieve higher performances.

A Corpus-based Evaluation of a Domain-specific Text to Knowledge Mapping Prototype

A Corpus-based Evaluation of Lexical Components of a Domainspecific Text to Knowledge Mapping Prototype

Development of a Conceptual Structure for a Domain-Specific Corpus

Research on the Knowledge Map of Combat Simulation Domain Based on Relational Database

KETM:A Knowledge-Enhanced Text Matching method

Keyphrase Based Evaluation of Automatic Text Summarization

Evaluating User Knowledge in Large Scale Online Knowledge Communities

Scalable Knowledge Graph Construction over Text using Deep Learning based Predicate Mapping

A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories

Evaluating Language Models for Knowledge Base Completion

Understood in Translation, Transformers for Domain Understanding

Improving Knowledge-Aware Dialogue Response Generation by Using Human-Written Prototype Dialogues.

Un mod{è}le de base de connaissances terminologiques

KoBE: Knowledge-Based Machine Translation Evaluation

A Knowledge-Injected Curriculum Pretraining Framework for Question Answering

What’s in an Index: Extracting Domain-specific Knowledge Graphs from Textbooks

AdaptKT - A Domain Adaptable Method for Knowledge Tracing.

Mapping the Knowledge Domain of Text Inferencing

Knowledge Translation: Extended Technical Report

Building a Large-Scale Knowledge Base for Machine Translation

Customized Information and Domain-centric Knowledge Graph Construction with Large Language Models