Abstract:Among existing knowledge graph based question answering (KGQA) methods, relation supervision methods require labeled intermediate relations for stepwise reasoning. To avoid this enormous cost of labeling on large-scale knowledge graphs, weak supervision methods, which use only the answer entity to evaluate rewards as supervision, have been introduced. However, lacking intermediate supervision raises the issue of sparse rewards, which may result in two types of incorrect reasoning path: (1) incorrectly reasoned relations, even when the final answer entity may be correct; (2) correctly reasoned relations in a wrong order, which leads to an incorrect answer entity. To address these issues, this paper considers the multi-hop KGQA task as a Markov decision process, and proposes a model based on Reward Integration and Policy Evaluation (RIPE). In this model, an integrated reward function is designed to evaluate the reasoning process by leveraging both terminal and instant rewards. The intermediate supervision for each single reasoning hop is constructed with regard to both the fitness of the taken action and the evaluation of the unreasoned information remained in the updated question embeddings. In addition, to lead the agent to the answer entity along the correct reasoning path, an evaluation network is designed to evaluate the taken action in each hop. Extensive ablation studies and comparative experiments are conducted on four KGQA benchmark datasets. The results demonstrate that the proposed model outperforms the state-of-the-art approaches in terms of answering accuracy.

Rtrl: Relation-Aware Transformer with Reinforcement Learning for Deep Question Generation

Reinforced Multi-task Approach for Multi-hop Question Generation

Exploring Question-Specific Rewards for Generating Deep Questions

Relation-Guided Pre-Training for Open-Domain Question Answering

Reinforced Dynamic Reasoning for Conversational Question Generation

Improving Neural Question Generation using Deep Linguistic Representation

TransferNet: an Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph

An Interpretable Reasoning Network for Multi-Relation Question Answering.

Boosting question answering over knowledge graph with reward integration and policy evaluation under weak supervision

Implicit Relation Linking for Question Answering over Knowledge Graph.

Relation-Aware Fine-Grained Reasoning Network for Textbook Question Answering

DRS: Deep Question Reformulation With Structured Output

RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees

Multi-hop Question Answering for SRLGRN Augmented by Textual Relationship Modelling

Action-Centric Relation Transformer Network for Video Question Answering.

Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds

Weak Supervision Enhanced Generative Network for Question Generation

Ask to Understand: Question Generation for Multi-hop Question Answering

R-3: Reinforced Ranker-Reader for Open-Domain Question Answering

QAGCN: Answering Multi-Relation Questions via Single-Step Implicit Reasoning over Knowledge Graphs

DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering