Abstract:Answering questions using pre-trained language models (LMs) and knowledge graphs (KGs) presents challenges in identifying relevant knowledge and performing joint reasoning.We compared LMs (fine-tuned for the task) with the previously published QAGNN method for the Question-answering (QA) objective and further measured the impact of additional factual context on the QAGNN performance. The QAGNN method employs LMs to encode QA context and estimate KG node importance, and effectively update the question choice entity representations using Graph Neural Networks (GNNs). We further experimented with enhancing the QA context encoding by incorporating relevant knowledge facts for the question stem. The models are trained on the OpenbookQA dataset, which contains ~6000 4-way multiple choice questions and is widely used as a benchmark for QA tasks. Through our experimentation, we found that incorporating knowledge facts context led to a significant improvement in performance. In contrast, the addition of knowledge graphs to language models resulted in only a modest increase. This suggests that the integration of contextual knowledge facts may be more impactful for enhancing question answering performance compared to solely adding knowledge graphs.

What problem does this paper attempt to address?

The main goal of this paper is to improve question answering systems based on pre-trained language models (LM) and knowledge graphs (KG) by integrating external knowledge facts. Specifically, the paper explores the following points: 1. **Problems and Challenges**: - Pre-trained language models perform well in language understanding but face difficulties in handling structured reasoning tasks. - Knowledge graphs can supplement language models in terms of structured knowledge, but effectively combining the two remains a challenge. 2. **Method Comparison**: - The paper compares the effectiveness of using only fine-tuned language models (LM) with the combination of language models and knowledge graphs (QAGNN method). - The QAGNN method uses the language model to encode the question and answer context, estimates the importance of knowledge graph nodes, and then updates the question-selected entity representations through a graph neural network (GNN). 3. **Experimental Results**: - Experiments show that incorporating knowledge facts into the question answering task significantly improves performance, while merely adding a knowledge graph results in a smaller improvement. - The method combining knowledge facts achieved better results on the OpenBookQA dataset, especially when additional knowledge facts were included. 4. **Research Significance**: - By enhancing existing methods, this research aims to advance the frontier of natural language understanding, particularly in structured reasoning. - This approach can be applied to various scenarios, such as virtual assistants, search engines, and educational tools, thereby improving user experience. Overall, this paper aims to explore how to better integrate language models and external knowledge to enhance the accuracy and context-awareness of question answering systems.

FusionMind -- Improving question and answering with external context fusion

Fusing Context Into Knowledge Graph for Commonsense Question Answering

QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering

Knowledge Graph Question Answering with semantic oriented fusion model.

Structured Knowledge Grounding for Question Answering

FeQA: Fusion and enhancement of multi-source knowledge on question answering

Local and Global: Temporal Question Answering Via Information Fusion.

JellyBell at TextGraphs-17 Shared Task: Fusing Large Language Models with External Knowledge for Enhanced Question Answering

Graph Reasoning Transformers for Knowledge-Aware Question Answering

Parallel Fusion of Graph and Text with Semantic Enhancement for Commonsense Question Answering

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph

Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge

TransKGQA: Enhanced Knowledge Graph Question Answering With Sentence Transformers

Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering

Efficient Question Answering Based on Language Models and Knowledge Graphs

Time-aware Multiway Adaptive Fusion Network for Temporal Knowledge Graph Question Answering

Knowledge Graph-augmented Language Models for Complex Question Answering

Graphusion: Leveraging Large Language Models for Scientific Knowledge Graph Fusion and Construction in NLP Education

QA-RAG: Exploring LLM Reliance on External Knowledge

Context Matters: Pushing the Boundaries of Open-Ended Answer Generation with Graph-Structured Knowledge Context

Context-aware Multi-level Question Embedding Fusion for visual question answering